Re: Very slow append rewrites file intermittently, why?

chrisp@xxxxxxxxx wrote:
Hi, I have an application that appends to 40mb files, intermittently
this append operation takes a long time (8-30 seconds) to append 300k
of data.

When watching the application with task manager I notice that the
application writes 40mb of data when this occurs. (the size of the
file I'm appending to)

To ensure I wasn't writing 40mb of data I wrote a test program.
(attached below) which opens a file in binary append mode, writes some
data, and closes it.

I tried the test app and always saw 20 I/O writes and 300,000 bytes.
The 20 I/O writes is a small puzzle. In testing there were always
two writes logged per call to fwrite(), even when writing 10 megabytes
at a time plus with large buffer sizes the bytes-written counter

My first thought when reading your post was that you were not opening
the file in binary mode and something in the C-library was rewriting
things.
However, you say you are using binary append mode and the sample
code does this.

While your test app looks fine try coding it using CreateFile(),
SetFilePointer(),
and WriteFile() to see if the problem is with the C library or the
operating system.

Marc

.

Relevant Pages

  • Appending to an indexed DataTable - performance degrades
    ... // Time the append operation ... It takes about 30,000 milliseconds to append this many rows with the ... believe that index maintenance time is proportional to the size of the ... back after the append operation, but I am wondering if anyone has any ...
    (microsoft.public.dotnet.framework.adonet)
  • Very slow append rewrites file intermittently, why?
    ... this append operation takes a long time to append 300k ... To ensure I wasn't writing 40mb of data I wrote a test program. ... int main(int argc, char *argv) ...
    (microsoft.public.win2000.file_system)
Loading