Re: ReadFileScatter and WriteFileGather

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

"Beverly Brown" <beverly.brown@xxxxxxxxx> wrote in message
news:%23Pmjry7RGHA.4688@xxxxxxxxxxxxxxxxxxxxxxx
Looking at the docs for these functions, I see that the buffers in the
array must be page-sized and page-aligned. I haven't quite figured out how
this is useful in a real-world example. Why would an application want
discontiguous page-sized, page-aligned virtual buffers?

Page-aligned page-size buffers can be passed down to bare metal without any
transformations, which is required for maximum efficiency. If you don't care
about this, you should not care about scatter-gather IO, either.

The usefulness of discontiguous buffers cannot be seen in a example that can
be posted here.

One example is a DB server that must cache disk data without using the OS
cache. While the cache is relatively free, you can read contiguous disk data
into contiguous buffers. But eventually the cache will fill up, and then
parts of it will be purged, and you suddenly cannot read contiguous disk
data into contiguous buffers, unless you grow the cache, or de-fragment the
cache, or issue multiple IO requests, or just read contiguous data into
discontiguous buffers. Do I need to explain why the latter option is the
best?

In order to get such an array, VirutalAlloc would need to be called
repeatedly requesting single pages each time.

This can be done, but this is exactly what should not be done. Instead, you
allocate a big contiguous chunk and maintain a free page list/bitmap and use
whatever free pages that happen to be in the list. You do not re-allocate
each time you do IO.

[...]

Also, if the buffers need to be page-aligned as the docs say, why bother
having a member specifying Alignment in the FILE_SEGMENT_ELEMENT
structure?

It is not a structure, it is a union. This element ensures that each element
is 64 bit aligned.

S


.

Relevant Pages

  • Re: Informix FAQ
    ... BTR is an attempt to quantify how much or little churning of the buffer ... That is deivided by the number of buffers in the cache and the ... The UBTR was a largely unsuccessful attempt to eliminate unused buffers from ... See you at the 2010 IIUG Informix Conference ...
    (comp.databases.informix)
  • RE: Informix FAQ
    ... portion of the cache was replaced. ... Here's my reasons for using pagreads and bufwrits. ... The other side of the BTR are the pagreads. ... Your pagwrits indicate that you wrote 2,620,000 buffers per hour to disk. ...
    (comp.databases.informix)
  • Re: does a CPU with hit-under-miss always imply that a microarchitecture has refill buffers?
    ... counting the number of refill buffers. ... buffers than the number of cache misses tolerated. ... Is this a CPU microarchitectural axiom? ... the small number of "refill buffers" is a bottleneck. ...
    (comp.arch)
  • Re: Increased Buffers due to patch 56e49d (vmscan: evict use-once pages first), but why exactly?
    ... All is about the increased amount of "Buffers" accounted as active while ... But I didn't get how this prefers buffers compared to cache pages (I ... This indeed sounds like the kind of workload that would only ... it might makes stream I/O benchmark score a bit because such workload ...
    (Linux-Kernel)
  • Re: Increased Buffers due to patch 56e49d (vmscan: evict use-once pages first), but why exactly?
    ... All is about the increased amount of "Buffers" accounted as active while ... But I didn't get how this prefers buffers compared to cache pages (I ... This indeed sounds like the kind of workload that would only ... filesystem metadata all the time. ...
    (Linux-Kernel)