Re: Some questions about sparse files
- From: "Alexander Grigoriev" <alegr@xxxxxxxxxxxxx>
- Date: Wed, 24 Dec 2008 08:33:07 -0800
1. You mentioned sort of key. Why can't the key be just a 64-bit file
offset? How do you pass data length to the clients?
2. Does your file needs to be persistent between sessions, needs backup?
3. You said circular queue. Does that mean number of blobs has an upper
limit? WHat happens when you need to overwrte a blob?
By the way, I don't see anything wrong with simply storing an index in the
start of file. That will also add persistence. Any overhead would be no
greater than filesystem overhead for sparse file access. Actually, it would
be much less.
"nickdu" <nicknospamdu@xxxxxxxxxxxxxxxx> wrote in message
news:AA991457-B503-4C1E-B163-3C72D49ED151@xxxxxxxxxxxxxxxx
Do you have another solution in mind?
I indicated the blob sizes are variable and I can't pick a max. Well
actually I am picking a max, but that should be an unreasonably large
value
that should be much larger than anyone needs. That's the beauty of using
sparse files, or so I gathered. There must be some reason why the OS
chose
sparse files for the NTFS change journal.
--
Thanks,
Nick
nicknospamdu@xxxxxxxxxxxxxxxx
remove "nospam" change community. to msn.com
"Alexander Grigoriev" wrote:
I think any alternate solution wil be more manageable than spare files.
You didn't say anything about blob sizes, persistency, etc.
"nickdu" <nicknospamdu@xxxxxxxxxxxxxxxx> wrote in message
news:ABA4112C-311F-48C1-A39E-9A62F4648620@xxxxxxxxxxxxxxxx
I have a couple questions regarding sparse files.
1. We're looking for an easy and efficient way to store blobs (array of
bytes) of data in some sort of circular queue and return some sort of
key
someone can use for later access to that blob. The blobs can very
varying
lengths. I was wondering whether sparse files would be a reasonable
approach
for this as it appears it's an efficient mechanism for storing messages
of
varying length (at least that's what I gather).
I guess you could also store each blob in its own file, but then I
think
the
overhead of creating a file per blob (may have millions) might be
costly.
I could also store all the blobs in a single "normal" (non-sparse)
file,
but
then I think the house keeping of walking the chain of blobs (most
likely
I'll need to chain them if I don't use a sparse file) might be costly
in
terms of performance and also adds to the code I'll have to write.
Though
I
guess NTFS is keeping its own list of sections of the file that contain
data.
2. If I want to copy a sparse file to other Windows machines running
NTFS
do
I need to write my own code to do that or does CopyFile() handle sparse
files
such that it only copies the parts of the file which contain data such
that
the copied file is exactly the same as the source file?
--
Thanks,
Nick
nicknospamdu@xxxxxxxxxxxxxxxx
remove "nospam" change community. to msn.com
.
- Follow-Ups:
- Re: Some questions about sparse files
- From: nickdu
- Re: Some questions about sparse files
- From: nickdu
- Re: Some questions about sparse files
- References:
- Some questions about sparse files
- From: nickdu
- Re: Some questions about sparse files
- From: Alexander Grigoriev
- Re: Some questions about sparse files
- From: nickdu
- Some questions about sparse files
- Prev by Date: Re: Some questions about sparse files
- Next by Date: DbgPrint log without debugger
- Previous by thread: Re: Some questions about sparse files
- Next by thread: Re: Some questions about sparse files
- Index(es):
Relevant Pages
|
