RE: Flash filesystem performance

It is possibly but not likely. I think that blocks are used for compaction by
the FAL.

We have noticed that if we put our device to sleep and then reset the device
after 5 seconds, the corruption does not occur. Then we shorten the time
beetwen sleep and reset and atfer we got to 1 second the corruption occured.
During these tests we are checking the STS pin of the flash and it is
sometimes active longer than 1 second so we are interrupting flash
operations. With the 5 seconds time we do not interrupt the flash and the
filesystem survives. This ought to be connected with some lowlevel routines
for erasing or writting to flash.

Currentlly I am not able to do a lot of these tests with TFAT system because
my boss wants the product fast and I must stick up with the IPSM filesystem
which works for us. We have also changed our application not to be dependent
on flash as much as before so the performance boosts up.

I would really like to see TFAT filesystem working so any finding would be
very appreciated.

Thanks, Jernej

"anonymous" wrote:

> I finally got around to looking at what's happening as far as writing FAT's.
> They are being put into different sectors through the process, but are not in
> different blocks.
>
> Nothing is getting erased as part of a mount process for me though. The FAL
> algorithm holds at least a couple of blocks privately for its use. I assume
> this allows it to know there is always something free during compaction.
>
> It will erase blocks on startup that aren't formatted correctly.
> (VerifySignatures in the sample strata FMD). Any chance that's somehow always
> seeing the signature as bad.
>
> "turnsek" wrote:
>
> > TFAT is very good filesystem when you are not powercycling a lot. But with
> > powercycling it gets corrupted very soon (about 2 hours to 10 hours of 1
> > minute cycles). I have checked erasing block ids and found out that after
> > mount four blocks are erased first: block 0 (first one in flash), block 191
> > (the last one in our flash), block 1 and 2. These gets erased first after
> > mount and then on the blocks are erased sequentially from the start to the
> > end (wear leveling techniqes). This is strange because something is allways
> > in these blocks.
> >
> > We are performing stress tests on flash when the powercycling is enabled (a
> > lot of writes and reads from flash).
> >
> > "anonymous" wrote:
> >
> > > I'm working with FAT16. I've got paired Intel strata flashes I'm working with
> > > at the moment. We've been using IPSM, but are evaluating a bunch of different
> > > flash options (mSystems, spansion, etc.) and TFAT seems to be the most
> > > appropriate step for non-Intel replacement of IPSM. I haven't started power
> > > cycling yet, but I will be soon. I'm trying to pass on the first try by
> > > learning from others trials rather than iterate.
> > >
> > > I'm going to go see where my two FAT tables end up on FAT16. Your suggestion
> > > that they may be in the same flash block sounds plausible, but should be
> > > fairly straightforward to check. I've just got to rebuild...
> > >
> > > "turnsek" wrote:
> > >
> > > > Hi!
> > > >
> > > > We are using recommended configuration for the TFAT flags, but we tried with
> > > > other settings too and no luck. We are not using FAT12 or 16, but FAT32. So
> > > > the flags are not the issue here.
> > > >
> > > > What if both FAT tables are in one erase block (256KB erase block on
> > > > 28F256J3C flash) and this gets damaged somehow? This would efectivelly kill
> > > > both of them. Isn't it? I have also suspected the low level driver, but after
> > > > I have examined more closely I couldn't find what is wrong. Even FMD from
> > > > WinCe5.0 which is much better written than FMD from WinCe4.2 is not working
> > > > to solve this power interruption problem. First I was shure that some erase
> > > > block operation is interrupted during power down and then on powerup after
> > > > mount this erase block is considered erased which is not. But in 5.0 version
> > > > this is fixed with signatures. So this could not happen anymore. If it is the
> > > > FMD then something else must be wrong.
> > > > I am not shure that interruption of the writes to the media presents the
> > > > problem, because TFAT should prevent that. So I am in the dark here.
> > > >
> > > > Are there any other filesystems beside TFAT, IPSM and FlashFX(last is
> > > > expensive)?
> > > >
> > > > Did you had some problems with TFAT during power down and power up cycles?
> > > > What kind and size of flash are you using? What are your power reserves etc.?
> > > >
> > > > Regards, Jernej
> > > >
> > > >
> > > > "anonymous" wrote:
> > > >
> > > > > Looking at the flags 00780034, a few questions:
> > > > >
> > > > > 1) bit 0x00100000 - FATFS_DISABLE_TFAT_REDIR. Doing this would make a FAT12
> > > > > or FAT16 less secure if I read things right. This is pushing the root
> > > > > directory down a level so that changes to it are transacted. By setting this
> > > > > flag and disabling the feature, changes to the root would be less secure.
> > > > > 2) bit 0x00040000 - FATFS_TRANS_DATA. This will slow down writes, but I
> > > > > wonder if it will make things better for you.
> > > > > 3) If compaction is the culprit, I wonder if increasing the number of
> > > > > reserved sectors/blocks would help. By default bootpart will reserve 2 or
> > > > > 0.25% of the disk. If you reserved say 4 or 8 sectors (since its most likely
> > > > > just reserving 2), I wonder if it will help.
> > > > > 4) If both FAT0 and FAT1 are corrupted, it really seems to indicate that
> > > > > something is wrong at or below the block driver level. FAT1 should be safe
> > > > > when FAT0 is being updated. If it isn't then that's a major focus point. The
> > > > > FAL isn't going to have both open and be doing updates to both at the same
> > > > > time, so what's really happening.
> > > > >
> > > > > "turnsek" wrote:
> > > > >
> > > > > > Ok, First our TFAT flags are 00780034 and registry keys are following:
> > > > > >
> > > > > > ; @CESYSGEN IF CE_MODULES_STRATAD
> > > > > > ; HIVE BOOT SECTION
> > > > > > ; StrataFlash block driver.
> > > > > > [HKEY_LOCAL_MACHINE\Drivers\BuiltIn\StrataFMD]
> > > > > > "Dll"="stratad.dll"
> > > > > > "Order"=dword:2
> > > > > > "Prefix"="DSK"
> > > > > > "Ioctl"=dword:4
> > > > > > "Profile"="MSFlash"
> > > > > > "IClass"="{A4E7EDDA-E575-4252-9D6B-4195D48BB865}"
> > > > > > "MemBase"=dword:b9300000
> > > > > > "MemLen"=dword:03000000
> > > > > > "SectorSize"=dword:200
> > > > > > "BlockSize"=dword:20000
> > > > > > "IsPairedFlash"=dword:1
> > > > > >
> > > > > > ; Override names in default profile
> > > > > > [HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\MSFlash]
> > > > > > "DefaultFileSystem"="FATFS"
> > > > > > "Name"="MSFLASH for STRATAFLASH"
> > > > > > "Folder"="Storage Card"
> > > > > > "AutoMount"=dword:1
> > > > > > "AutoPart"=dword:0
> > > > > > "AutoFormat"=dword:0
> > > > > >
> > > > > > ; Keep FATFS from trying to shadow \Windows
> > > > > > [HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\MSFlash\FATFS]
> > > > > > "Dll"="fatfsd.dll"
> > > > > > "MountFlags"=dword:0
> > > > > > "Flags"=dword:00780034
> > > > > > "Util"="fatutil.dll"
> > > > > > "Paging"=dword:1
> > > > > > "CacheSize"=dword:0
> > > > > > "FormatTfat"=dword:1
> > > > > >
> > > > > > [HKEY_LOCAL_MACHINE\System\StorageManager\AutoLoad\MSFlash]
> > > > > > "DriverPath"="Drivers\\BuiltIn\\StrataFMD"
> > > > > > "LoadFlags"=dword:1
> > > > > > "Order"=dword:0
> > > > > > "BootPhase"=dword:0
> > > > > >
> > > > > > We have PXA255 based platform and we are totally dependent on flash
> > > > > > filesystem. We were using TFAT before and after extensive powercycle testing
> > > > > > we have concluded that is not stable. We even tried FMD driver with WINCE5.0
> > > > > > which is much better (erase block signatures added etc.) than FMD in
> > > > > > WINCE4.2, but still not good enough. I think something must go wrong with FAT
> > > > > > tables updating (both of them are getting lost or just pšart of them). They
> > > > > > become corrupted after overnight testing (lost files, missing directories,
> > > > > > sometimes even deadlock of the filesystem). You can see my previous posts. I
> > > > > > am afraid that there is a problem in FAL library which is not documented at
> > > > > > all. The problem is most likely connected with compaction thread. I think
> > > > > > TFAT is good for battery backed up devices which have enough time to
> > > > > > gracfully shut down the filesystem and never interrupt the flash internal
> > > > > > routines, but in embedded devices without battery backup (like our) things
> > > > > > are not so simple. Power cycle could easily interrupt flash internal routines
> > > > > > and driver hasn' t enough time to shut down (ups problems:)).
> > > > > >
> > > > > > With IPSM the filesystem is stable (even pure hardware resets witn no
> > > > > > powerdown signaling can't corrupt it), but it's slow :(.
> > > > > >
> > > > > > If you have some ideas how to stabilise TFAT I will be very interested.
> > > > > >
> > > > > > Thanks, Jernej
> > > > > >
> > > > > >
> > > > > > "anonymous" wrote:
> > > > > >
> > > > > > > Well, its not open source as of yet, so nothing to be done unless you
> > > > > > > complain to Intel.
> > > > > > >
> > > > > > > Let's get back to what is in our control. What were you using for FATFS
> > > > > > > flags and registry keys under TFAT ?
> > > > > > >
> > > > > > > I'm looking a new flash options not supported under PSM, so looking at TFAT,
> > > > > > > but there seems to be a body of posts that TFAT isn't robust enough for
> > > > > > > folks, but there are a lot of options and I'm not clear that all necessarily
> > > > > > > default to the most conservative state.
> > > > > > >
> > > > > > > If folks are having problems with TFAT getting corrupted, can we work as a
> > > > > > > community to figure out what's not working ?
> > > > > > >
> > > > > > > "turnsek" wrote:
> > > > > > >
> > > > > > > > Ok, I understand factor 1 or 2 maybe, but 10 times slower than TFAT, please.
> > > > > > > > This must be something wrong in the IPSM code. As I understand, filesystem
> > > > > > > > structure should be kept in RAM and searched quickly and when the file is
> > > > > > > > found then data should be read out from flash. Here seems to be everything is
> > > > > > > > kept on flash :(.
> > > > > > > >
> > > > > > > >
> > > > > > > > "anonymous" wrote:
> > > > > > > >
> > > > > > > > > I don't think any of us can answer this until Intel goes open-source with
> > > > > > > > > PSM, but you're results are not unusual. PSM does not perform as fast as
> > > > > > > > > FAT-based systems.
> > > > > > > > >
> > > > > > > > > Try opening a file with write priviledges in a subdirectory three or four
> > > > > > > > > levels down and closing it and see how long it takes. Quite an eye-opener.
> > > > > > > > >
.

Relevant Pages

  • Re: TFAT stability
    ... you can see that disk-like 512-byte sectors are mapped plain linearly to flash memory. ... TFAT knows nothing of flash blocks. ... We have turned off AutoScan and the platform seems to be stable. ... I think the filesystem moust be unmounted during ...
    (microsoft.public.windowsce.platbuilder)
  • Re: TFAT stability
    ... So you think there is no solution to stabilize the filesystem? ... you can see that disk-like 512-byte sectors are ... > mapped plain linearly to flash memory. ... TFAT knows nothing of flash blocks. ...
    (microsoft.public.windowsce.platbuilder)
  • Re: TFAT stability
    ... if you look at how the msflash block driver is implemented, you can see that disk-like 512-byte sectors are mapped plain linearly to flash memory. ... TFAT knows nothing of flash blocks. ... I think the filesystem moust be unmounted during ...
    (microsoft.public.windowsce.platbuilder)
  • LogFS take five
    ... Add LogFS, a scalable flash filesystem. ... The two main problems of JFFS2 are memory consumption and mount time. ... Flash behaves significantly different to hard disks. ...
    (Linux-Kernel)
  • Re: [patch] ext2/3: document conditions when reliable operation is possible
    ... of your ext3 + flash card issue - is it the ftl stuff doing out of order IO's? ... The problem is that flash cards destroy whole erase block on unplug, ... this isn't a filesystem specific cliam; ...
    (Linux-Kernel)
Loading