Enable Memory Restriction for Lookup Transformation

Tech-Archive recommends: Fix windows errors by optimizing your registry



Hi,

We can enable memory restriction (limit cache size) for “Lookup
Transformation" in Data Flow Task, can someone tell me how exactly it
effects the performance of the package?

I’m fetching 35 million records from source database and having lookup
against table which has almost 45 million records. So when I’m
executing the package, for first couple of minutes (approx 5-10 mins)
it caches the lookup table and then fetches the records from source
db, perform the real lookup and send it to destination.

Can someone tell me if I enable memory restriction on lookup
transformation how exactly it will perform lookup on record set which
is not cached and is there any risk in enabling the memory
restriction, I mean it may not be able to perform lookup up on non
cached data or something like that, is there any performance benefit/
drawback while enabling it?

As I'm new to SSIS & so may be my question sound very basic, silly or
repetitive so sorry for that.

Thanks in advance.

Cheers,
Manan
.



Relevant Pages

  • Re: code optimization help
    ... interpolation lookup table for speed. ... reverse map the coordinate into the source image ... transformation equations are complex and take a long time to ... You are converting the interpolation to a lookup table as ...
    (comp.soft-sys.matlab)
  • Re: SSIS and Staging Tables
    ... If the lookup is not a match in the destination table ... The error output then runs to an OLEDB transform. ... the input columns in your case to an update statement. ... understanding of something fundamental in the Data Flow Task process. ...
    (microsoft.public.sqlserver.dts)
  • Re: DTS Lokup %
    ... My Lookup Query looks like this ... Allan Mitchell MCSE,MCDBA, (Microsoft SQL Server MVP) www.SQLDTS.com - The site for all your DTS needs. ... > It works in ExecuteSQL task but it doesnt work in Data Transformation Lookup.. ... I need specifically in lookup because while transformation I have to find corresponding value.. ...
    (microsoft.public.sqlserver.dts)
  • Re: DTS Passing Null values using Oracle
    ... The flow is something like this... ... > using activex script on the transformation to passing all the values ... > through a lookup to do the update. ... For each of the fields that I was passing ...
    (microsoft.public.sqlserver.dts)
  • Re: SSIS and Staging Tables
    ... if there is any way (within the Data Flow Task) to ... Lookup transform. ... The error output then runs to an OLEDB transform. ... the input columns in your case to an update statement. ...
    (microsoft.public.sqlserver.dts)