Re: FAT32 or NTFS?

Tech-Archive recommends: Fix windows errors by optimizing your registry

From: cquirke (MVP Win9x) (cquirkenews_at_nospam.mvps.org)
Date: 05/13/04 

Date: Thu, 13 May 2004 21:55:48 +0200

On Wed, 12 May 2004 04:41:52 GMT, "Alexander Grigoriev"

>Consider expanding a file. Before any data can be written to a file, new
>clusters has to be allocated, free clusters bitmap modified, directory entry
>modified to reflect new extent allocated (although the file length won't be
>written immediately to the directory entry). Only after that you will be
>able to write new data to a file.

>You're more likely to have a file with garbage data, than to get valid data
>truncated. I'd say the latter case is close to impossible.

Just thinking of prior experience with FATxx where files-in-progress
are so often recoverable as lost cluster chains (where dir entry has
either been deleted or not yet created) and length mis-matches (where
cluster chaining implies a length at odds with dir entry).

As I understand it, the dir entry is created when the file is opened,
but only when the file is closed, is the length set to correct value.
In between these events, data is indeed written to the cluster chain,
and FAT updates happen at this time.

Presumably a similar sequence of events occurs with NTFS; dir entry
created if it does not yet exist, data is written to the current data
run in the data stream, free cluster bitmap hopefully updated; then
when done, dir entry length value is updated.

I take it that NTFS's transaction logging starts when the file is
opened and ends (and commits) when the file is closed.

Good programming practice may prefer to hold "real" files open for as
short a time as possible. The longer the file is held open, the more
likely it is to get messed up, and it also acts as a sand-bar or
skip-file for defrag, backup, etc.

But as opening and closing files are slow, hoggy tasks, you want to do
multiple small updates as a single operation rather than have to open
and close the file each the time. The solution may be to hold a
temporary file open just for the bits you are working on, and then
commit these into the "real" file when saved.

In between opening the file and saving it, the "real" file may be left
closed, though some checks must be done to resolve situations where
something else has modified the file.

>From what I see, it's mainly log files and System Restore stuff that
commonly gets broken on bad exits from XP. Presumably these file are
held open, but (in the case of log files) flushed frequently?

>If you want your data consistent, you have to implement your own transaction
>system for your application files. You can use FILE_FLAG_NO_BUFFERING, or,
>for special cases, FILE_FLAG_WRITE_THROUGH.

It's been a long while since I worked at that level ;-)

>------------ ----- ---- --- -- - - - -
  Our senses are our UI to reality
>------------ ----- ---- --- -- - - - -

Relevant Pages

  • w2k cluster stopps serving resources unexpected.
    ... There are only fileshares on that cluster. ... GumSendUpdateOnVote: ... LogpReset entry... ... >exit, returning 0x00000000 ...
    (microsoft.public.win2000.advanced_server)
  • Re: Cluster Error 1080
    ... but I dont find the entry in the cluster.log. ... the Windows 2003 cluster the nodes are domain controllers. ... DSAccess for "configuration domain controller" was pointed to an old ... > Generic Service registry replication key entry. ...
    (microsoft.public.windows.server.clustering)
  • Re: directory size
    ... An MFT entry is 1k. ... >>A directory will initially take one cluster on the disk, ... > long file name would not result in extra entries, ...
    (microsoft.public.windows.file_system)
  • w2k cluster stopps serving resources unexpected.
    ... HP DL380G3s 2.5GB RAM, Active/Passive cluster, HP ... HP MSA100 SAN serving 1200 ... >LogpMountLog: Entry ... >>exit, returning 0x00000000 ...
    (microsoft.public.win2000.advanced_server)
  • Re: [PATCH 2/2] ext2: Avoid rec_len overflow with 64KB block size
    ... into 16 bits we have for entry lenght. ... Just to clarify this is only changes the directory entries format on ... The Documentation/filesystems/ext2.txt doesn't have the ext2 format ...
    (Linux-Kernel)