Re: asynchronously writing to a file, a cheap way?
- From: nickdu <nicknospamdu@xxxxxxxxxxxxxxxx>
- Date: Tue, 4 Aug 2009 11:08:01 -0700
As I mentioned, I don't own the application. In cases where I don't own the
application I don't like spinning up my own threads. I was hoping I could
just go the route I suggested in my original post. Remy said that my
proposal was not an asynchronous method, and while I'll agree that it's not
your typical asynchronous route for writing to a file, it still seems to me
that "lazy" means that the OS is going to take care of writing this out at
some other point in time. Is this not true? Can someone further explain
what "lazy" means with respect to the docs for UnmapViewOfFile()?
Also, what happens if I do follow the steps I suggest in my original post
and my application crashes abruptly after step 7? Do the changes end up
making it to disk?
What happens if I start an asynchronous write using async IO and my
application crashes right after issuing the asynchronous write?
As you can probably tell I'm trying to come up with a way for me to provide
a library (set of routines) to an application such that it can write messages
(/stream of bytes) with very low overhead/latency and those events will be
persisted regardless of whether the application encounters a fault AFTER
calling my API. It probably sounds like I'm suggesting something like
queuing, and that might very well be true. I'm not interested in MSMQ for
several reasons which I won't go into (unless someone needs to know).
I was hoping I could, making use of shared memory, quickly share my messages
(stream of bytes) with another process and leave it up to the other process
to handle doing something with the bytes. While this in itself might not be
terribly difficult there is one other goal which I think does make it more
difficult. If this second process which processes the stream of bytes is not
running the application which wote the data should still be able to write the
data and it should be persisted in some fashion such that when the processing
application executes it can access that data and process it.
So yes, it is like persistent queues, though I would like in memory
performance when the processing server is running. I guess I'm looking for
something like named pipes where the pipe is backed by a file and the server
end of the pipe doesn't need to be running. Does anything like this exist?
--
Thanks,
Nick
nicknospamdu@xxxxxxxxxxxxxxxx
remove "nospam" change community. to msn.com
"m" wrote:
???.
Whether or not the IO can be performed by the driver(s) / HW in parallel, is
by no means a good way of deciding whether the application should use
overlapped IO. This decision needs to be made according to the design of
the application. Consider that it makes little sense for Apache server to
use overlapped IO when sending data to the client, but it makes a lot of
sense in IIS (IIRC 5.0 - before they moved to KM)
For the OP:
I you already have a thread pool or IOCP that you can use t handle the
completion notifications (and or work items to resize the file), then using
overlapped IO will save you at least the cost of a thread. If you don't
already have a thread pool or IOCP that you can use, then for a single file,
the cost of overlapped IO versus sync IO on a dedicated thread will be
similar; but if you will need to expand to multiple files in the future,
overlapped IO will behave much better.
"Matt" <none@xxxxxxxx> wrote in message
news:OY%23eg9k5JHA.3304@xxxxxxxxxxxxxxxxxxxxxxx
An overlapped write at the end of the file becomes a syncronous write...
The best way is to queue it off to a seperate thread and do the write
there.
"Remy Lebeau" <no.spam@xxxxxxxxxxx> wrote in message
news:OUOmtq84JHA.1432@xxxxxxxxxxxxxxxxxxxxxxx
"nickdu" <nicknospamdu@xxxxxxxxxxxxxxxx> wrote in message
news:72A28E05-6EB6-41B3-975E-E2B4C59E35C4@xxxxxxxxxxxxxxxx
I would like to write a buffer to a file asynchronously so that the
thread which generated the buffer is not blocked while the write
is occurring.
Use overlapped I/O or IOCP for that. They are both designed for
asynchronous I/O, and can be used with files.
I was thinking that I could:<snip>
That is not asynchronous writing, not even close.
The docs say that the OS will handle writing to the file in a lazy
fashion.
That is completely separate from having your code write data to a file
handle asynchronously.
what I really want is a way to write (only) to a file asynchronously.
Look at the lpOverlapped parameter of WriteFile(), or at the lpOverlapped
and/or lpCompletionRoutine parameters of WriteFileEx().
--
Remy Lebeau (TeamB)
- Prev by Date: Re: Run a program with no elevation from a program run as admin
- Next by Date: Interlocked statements on mapped memory
- Previous by thread: Re: ETW trace question
- Next by thread: asynchronously writing to a file, a cheap way?
- Index(es):
Relevant Pages
|
