Re: App Performance help
- From: "Nick Malik [Microsoft]" <nickmalik@xxxxxxxxxxxxxxxxxx>
- Date: Tue, 16 Aug 2005 07:59:20 -0700
Like I said, I don't know what unzip library you are using. Look into the
code in the library to see which is better at handling memory. In general,
you will get more performance by keeping things in memory, but you may get
better memory management by throwing things to disk.
--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik
Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
"Chris" <Chris@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:37F61631-2032-4FFC-BB32-0988118BF583@xxxxxxxxxxxxxxxx
> Each of my zip files has a single file and the file is about 200k with
> about
> 7000 rows. I'll have to loop abou 10 zip files. Should I unzip to memory
> orunzip the file to a folder then read the file and append each file to
> the
> new file created?
>
> "Nick Malik [Microsoft]" wrote:
>
>> It would work for a small set of files. Problem is that your plan is
>> terrible on memory. Strings are immutable in .Net which means that you
>> create a string for a file, create a string for another file, then throw
>> them both away as soon as you have a string for the two files together.
>>
>> If you are simply appending, then use a textreader and textwriter.
>>
>> Psuedocode
>> open the source zip
>> get a list of contained files
>> open the destination file for append
>> for each file in the zip
>> unzip the file (into memory?) I don't know what library you are
>> using
>> open the input file stream
>> while not eof(input file stream)
>> read a block of text (say, about 20K?)
>> write the block of text to the destination file
>> end while
>> close unzipped file (and clean up, if necessary)
>> next file
>> close destination file
>>
>>
>> This will scale to situations where the zip contains large individual
>> files
>> as well as when the zip contains large numbers of files.
>>
>> --
>> --- Nick Malik [Microsoft]
>> MCSD, CFPS, Certified Scrummaster
>> http://blogs.msdn.com/nickmalik
>>
>> Disclaimer: Opinions expressed in this forum are my own, and not
>> representative of my employer.
>> I do not answer questions on behalf of my employer. I'm just a
>> programmer helping programmers.
>> --
>> "Chris" <Chris@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
>> news:CC87915E-8DA9-4140-8596-FB192505ED8E@xxxxxxxxxxxxxxxx
>> > Hi,
>> > I am developing an app that will unzip several zip files and append the
>> > contents of the zip files to one text file. The zip files contains one
>> > text
>> > file with a standard format but the size of the file will be an average
>> > of
>> > 100k to 300k with at least 7000 rows.
>> >
>> > Could I use a simple string (Dim s as string)? So first I would unzip a
>> > file, read its contents, store it in the string then unzip the second,
>> > append
>> > it's contents to the same string ........after all the files are
>> > unzipped
>> > I
>> > then write the contents of the string to a text file. Is that the
>> > proper
>> > way?
>> >
>> > Thanks
>>
>>
>>
.
- References:
- App Performance help
- From: Chris
- Re: App Performance help
- From: Nick Malik [Microsoft]
- Re: App Performance help
- From: Chris
- App Performance help
- Prev by Date: Re: qbasic to vb.net conversion help. please.
- Next by Date: Forms Databinding Question
- Previous by thread: Re: App Performance help
- Next by thread: Re: App Performance help
- Index(es):
Relevant Pages
|
