Re: SISS vs Biztalk Large data

Hello,

I am too using SSIS instead of BizTalk to Process Large XML and Flat Files
sizing in few hundred MB's, in messaging scenarios of my project.

Through SSIS you can pick up XML or Flat Files from local or FTP folders and
Transform them n Push to some webservice or SQL database or some local
folders.

But, the Support for generating XML File or CSV file is not provided out of
the box n u have to write proper code for converting ur Relational data to
XML or CSV file, whatever is the case

You may find this article on how to deal with Flat files in SSIS useful:

http://aspalliance.com/889_Extracting_Data_from_a_Flat_File_with_SQL_Server_2005_Integration_Services

Though its pushing data to SQL Server you can convert the same to XML from
SQL server or create a text file out of the relational data.

You will have to code a lot manually for say if you want to generate XML out
of relational data as that things are not available out of the box n u have
to take care if you want data in XML or Flat File Format


Let me know if you want more info on this.

Thanks,
Sachin R. Chavan.


"Selvan" wrote:

Francis,

Thanks for your sugession, I just wonder how your file will be polled by
SSIS?
Do we have any better article/tutorial for Flat-to-flat scenario?.

Thanks again

Regards
Selvan.

"Francisco A. Gonzalez" wrote:

Hi,
SSIS is your tool.
I am a BizTalk guy, and so many times I use SSIS for scenarios like yours.

You can get the flat file, do look ups and transformations, do error
handling at record level, and convert the format to different flat files
formats.


BUT, it will for sure be faster than BizTalk but depending of RAM and your
skills with task and transformations you can get those few seconds that you
mention.

cheers.

Francisco A. Gonzalez
fgonzalez@xxxxxxxxxx



"Selvan" <Selvan@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:5546D8FF-9943-4FFB-A3D4-8525A2F02B1D@xxxxxxxxxxxxxxxx
Geeks,

I am a Biztalk guy, I receive a flatfile of 1GB approximatly everyday and
have to transform another flatfile format with in 30 mins of time. As for
as
i know flatfiles are converted to xml in biztalk which results 3 times
bigger
than the souce flatfile in biztalk databases. From various articles i
understood that SISS in SQL 2005 does it in few seconds. Can i transform
one
format data to different format flatfile in SISS?.
I just need to Group, Sum the records (I know biztalk map cannot play with
(1 GB *3 =3 GB) such a large file.

Apart from that i need a record level error handling, if one record
failed
for some reason say lookup not found or data error, can SISS log the error
and skip to
the next record?.

Since i am new to SISS do you have any suggession pls?

TIA,
Selvan.



.

Relevant Pages

  • Re: Receiving messages via http
    ... > You'll probably want to use the FTP adapter for that, or a web service ... >> data from XML and flat files but I am looking to receive data ... is there any way that BizTalk can access a remote ...
    (microsoft.public.biztalk.general)
  • Receiving messages via http
    ... data from XML and flat files (CSV etc) but I am looking to receive data via ... is there any way that BizTalk can access a remote XML ...
    (microsoft.public.biztalk.general)
  • Re: Maxed CPU on file adapter xml receive pipeline
    ... > others suffering from maxed CPU and eventually having to terminate BizTalk ... > This was really surprising as I have processed flat files using the flat ... > With XML inflation, they were in the order of 80mb in size. ...
    (microsoft.public.biztalk.general)
  • Re: 200MB and bigger flat files
    ... Are you using BizTalk 2006? ... I have been able to process 200 mb flat files through BizTalk 2006 ... I had sufficient memory on the BizTalk server, ... Server is bottleneck. ...
    (microsoft.public.biztalk.server)
  • RE: how to import an xml file to sql
    ... By reading in the xml with excel or flat files in pieces here is what I got ... in sql scripts: ...
    (microsoft.public.sqlserver.datawarehouse)