Re: NTFS block allocation policy (how does it work?)
From: Arne Ludwig (arne_at_citde.net)
Date: 02/28/05
- Next message: Udo Lenz: "Re: Is Athlons IOMMU supported?"
- Previous message: AF: "Re: Generic / text only printer problem"
- In reply to: Pat [MSFT]: "Re: NTFS block allocation policy (how does it work?)"
- Next in thread: Alexander Grigoriev: "Re: NTFS block allocation policy (how does it work?)"
- Messages sorted by: [ date ] [ thread ]
Date: 28 Feb 2005 04:56:16 -0800
The question to me remains when the actual zeroing occurs. If it
occured immediately after SetEndOfFile then the SetFileValidData call
would be quite useless since the 2nd argument to the call needs to be
between the current valid data length (0 on a new file) and the file
size.
So I would interpret Alexander to mean that the actual zeroing is
delayed until after the next access to "new space" after current valid
data length.
Correct?
PS. For whoever cares: My original problem seems to be improved by a
strategy of exponential file extension, e.g. allocating chunks in
powers of two. This often creates a file with a managable number of
fragments. The drawback of course is the increased complexity of the
writing code. A hint to NTFS would have been much handier.
"Pat [MSFT]" <patfilot@online.microsoft.com> wrote in message news:<#9vPZBVHFHA.2280@TK2MSFTNGP15.phx.gbl>...
> When you call SetEndOfFile() the blocks get zero'd. Just calling
> SetFilePointer() beyond the end of the file won't trigger it and you will
> see the behavior that you saw. You will also see that if you if the file is
> marked as Sparse or (I think) Compressed.
>
> Pat
>
>
>
> "Alexander Grigoriev" <alegr@earthlink.net> wrote in message
> news:%23LtANFOHFHA.3628@TK2MSFTNGP15.phx.gbl...
> >I observed that in XP the actual zeroing is delayed until you actually try
> >to access the file part beyound written. This is what Valid Data Size is
> >for.
> > SetFileValidData can be used to avoid zeroing altogeter, even if the file
> > is not written over.
> >
> > "Pat [MSFT]" <patfilot@online.microsoft.com> wrote in message
> > news:u9s94xIHFHA.3440@TK2MSFTNGP10.phx.gbl...
> >> NTFS zeros the file whenever you extend it. If you just use WriteFile()
> >> to extend the file, the same zero'ing occurs. This is for C2 security
> >> requirements, to prevent folks from extending their files over other
> >> peoples files and taking control of the data.
> >>
> >> If you extend a file a little at a time, then for every extension:
> >> Seek to cluster location
> >> Zero out enough space for the addition
> >> Seek back to the start of the write
> >> Write the file data
> >>
> >> (so, 2 seeks for the write)
> >>
> >> Also, if something else has written to the adjoining cluster in the
> >> interim since the last write, then the file will fragment. This happens
> >> all the time in logging applications.
> >>
> >> So, if you pre-allocate the file, you only pay for the zeroing 1 time.
> >> This will prevent any other files from intruding on your nicely
> >> sequential allocation and optimize write speeds. If you only pay the
> >> price once, then the the runtime performance is the same as bypassing the
> >> zeroing. The clusters do not get zero'd until you call SetEndOfFile().
> >> So the order is:
> >>
> >> CreateFile
> >> SetFilePointer (size of file you intend to create)
> >> SetEndOfFile (commit the file)
> >> SetFilePointer(beginning)
> >> start writing
> >>
> >> No more zeroing will occur unless you overrun the file size allocated.
> >> Also, fragmentation will be minimized. The zeroing happens very, very
> >> fast b/c there is no seeking. A single seek to the start of the file,
> >> then sequential writes so the writes occur at the disk speed.
> >>
> >> You can bypass the zeroing of files by using the SetFileValidData().
> >> This requires an elevated privilege to run (SE_MANAGE_VOLUME_NAME), so
> >> depending on the security context of your application you may have
> >> problems; also that is only available on WinXP and later so if you are
> >> targeting Win2k systems it won't work. Whereas the SetFilePointer can be
> >> executed by any user context that has access to the file. Since (from
> >> your description) the process creates the file as well, this won't be a
> >> problem.
> >>
> >> Pat
> >>
> >>
> >> "George M. Garner Jr." <gmgarner@newsgroup.nospam> wrote in message
> >> news:eU9bWe9GFHA.3484@TK2MSFTNGP12.phx.gbl...
> >>> Pat,
> >>>
> >>>> If you know at least the approximate size of the file, the best thing
> >>>> to do would be to pre-allocate the file, then go back and write to it.<
> >>>
> >>> Could you be more specific as to how to allocate the file. I see the
> >>> function SetFileValidData. But that assumes that you already have
> >>> allocated the file. Elsewhere I have read that one should not use
> >>> SetFilePointer(Ex) because of the added overhead of writing zeros to the
> >>> file. Is there a way to allocate the file without incurring the
> >>> overhead of immediately zeroing out the clusters. I know the pagefile
> >>> does this but I do not see an api that permits it.
> >>>
> >>> Regards,
> >>>
> >>> George.
> >>>
> >>
> >>
> >
> >
- Next message: Udo Lenz: "Re: Is Athlons IOMMU supported?"
- Previous message: AF: "Re: Generic / text only printer problem"
- In reply to: Pat [MSFT]: "Re: NTFS block allocation policy (how does it work?)"
- Next in thread: Alexander Grigoriev: "Re: NTFS block allocation policy (how does it work?)"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
