Re: Need Help deleting record from text file

"Dave O." <nobody@xxxxxxxxxxx> wrote in message news:e5pT7DRAIHA.4592@xxxxxxxxxxxxxxxxxxxxxxx

Why so defensive? All I said was that for some operations
a database is faster than flat files and for some operations
it is slower.

What you actually said was that retrieval will be faster, clearly implying that retrieval will *always* be faster, whereas in fact that is not necessarily so. In many circumstances retrieval of data from a "home grown" data file will actually be faster than from a general purpose commercial database. It's horses for courses.

If you are going to be so dogmatic that you cannot accept
that then I suspect that all your opinions are probably
questionable.

Depends what you mean by "questionable". Of course all my opinions are open to question. I welcome it. But I also reserve the right to question your own opinions. Do you have a poblem with that? It appears that you do.

If you are going to do a binary search of an array then
you need the whole thing in memory, or you need to
load it in fragments which will be slow and messy

Have you actually thought about what you've just said? You don't need to "load it in fragments which will be slow and messy". You don't need to load it at all! It's already loaded! The data is in already memory! It contains the names of files in a specific folder which were obtained using the standard GetFirstFile / GetNextFile method. It is read into memory by the FindNextFile API and is immediately transferred filename by filename into a standard VB String array by the VB code the OP is using. Why the *?#* would he want to save it into a database so that the database could perform some magic indexing of the data and then *#&#?%* read it back! That would be *slower*, not faster!!!

Of course there is no magic to databases, but you can
treat them as black boxes which might as well be magic

You can treat your car as a black box (or perhaps some other coloured box!) that might as well be magic. In fact some people do. But that doesn't make them magic!

the trick is obviously indexing . . .

Ah. Right. Indexing. Never thought of that one. No wonder these things are magic. Indexing eh. Couldn't possibly do magic tricks like that in VB code. Yikes!

which allows complex queries to be answered far
faster than any search on a flat file

I never mentioned a "flat file". I never once suggested that the OP should save his data to disk using a "flat file" unindexed arrangement. In fact I never mentioned him saving his data to disk in any format at all. The data I was helping the OP to deal with is data that is already in memory. That's what he asked for. If he later wants to use methods to save various items of data to disk then he'll have lots of choices. A "standard database" is just one of those choices, as is an unindexed "flat file". But they are not the *only* choices!

again obviously this only refers to large datasets, for a
few hundred items a flat file will be faster but we are
dealing with sets of several thousand items so the
hypothetical examples will be larger sets where the database has the clear advantage.

Again, you keep mentioning this "flat file" business that you seem to have picked out of thin air. I never once suggested anything to the OP along the lines of him using a flat file. Besides, are you seriously suggesting that the only way you can save data to disk in a form that is readily and rapidly retrievable is by using one of the standard databases? Do you really think that this indexing business is so "magic" that it cannot be done using standard code? Admittedly a nice "ready made" database does make it relatively easy for people to store and retrieve data fairly rapidly without needing to write too much code or to worry about the methods used by the "black box" to perform such feats, and I certainly would not suggest that they should not be used, but I think that for many tasks a "home grown" data format would be as good as, and in some cases better than, a standard database. And for anyone learning to program I would suggest that creating and using your own data formats in VB code (or any other code, for that matter) would be extremely useful, even if only for the fact that it would enable them to more easily understand how these "black boxes" of yours actually work.

The OP is as you say reading in the data to an array - He
then needs to store it so he can compare the folder content
with what is backed up, for that he needs a record of prior
content and a record of what's archived which will be either
the flat file or the database

There you go again! "Either the flat file or the database"! Have you no imagination or creative skill?

so a database will speed it up because he does not have to
read in the array of existing content and previous archive
records because all that can be left unloaded but accessible
in the database.

It is not possible to have it both "unloaded" and "accessible"! In order to access a piece of data on a disk then that piece of data needs to be loaded from disk. You can't just send a monkey out to crawl around the disk and report by telepathy what he finds! Who do you think wrote these "database" programs? They are not magic!

Also once you have a rough idea of what you are doing
coding for databases with VB6 is a doddle . . .

I'm sure it is. Much easier than creating your own general purpose data format, that's for sure. And I'm not really knocking them. But I am saying that in many cases, especially in fairly straight forward cases like this, there are other perfectly acceptable ways of achieving your aims that will result in code that works just as quickly and that will enable you to learn quite a lot of useful stuff into the bargain.

while maintaining ones own index of flat file content and devising a suitable random access method to get at the bits you need is a lot more complex.

Of course it is often more complex (at least initially until you've written the main code) but in many cases it is very simple and entirely suitable. In fact for someone who has never used a standard database before, and especially for fairly straight forward tasks, devising your own data structure and creating your own data file can be easier and quicker than learning to "drive" a standard database that was written by someone else.

You keep mentioning this "flat file" business, as though there was only one very restricted way of handling data on a disk! What do you think the data of a standard database is when it is sitting on the disk? It is just a lump of raw data, that's all. A database on disk is just a "flat file" of data. It's the way that the database "black box" code treats that data that makes it special. The only difference between a standard database file and a data file that you create yourself is that in a standard database all the work has already been done for you! That's fine of course, and it makes them fairly easy to use.But it doesn't make them something "magic" that cannot be achieved by any other means, and it doesn't always make them the most suitable option.

Mike




.

Relevant Pages

  • Re: Just say no to threads [Was: Software architecture]
    ... they knew there was going to be a database in the app. ... Now my colleague just spent a couple/three weeks designing an SQL ... Turns out they have a flat file of sample information, ...
    (comp.object)
  • RE: Cisco LEAP
    ... aren't meant to work this way; that's largely why database applications work ... For that, sure, a flat file is faster. ... Network with over 10,000 of the brightest minds in information security ... most highly-anticipated industry event of the year. ...
    (Pen-Test)
  • Re: Portugal and Azores Ceres question
    ... is the advantage of Access over a flat file ... I cannot see a stamp database requiring a deeper platform. ... Why risk the typos and the time to query a table like that? ... The report generator in Access is a 2 pass processor. ...
    (rec.collecting.stamps.discuss)
  • RE: Cisco LEAP
    ... he's going to want to do the kind of customizing that Inktomi did just to ... >> opening a graphic or text file completely into memory to ... >> For that, sure, a flat file is faster. ... > and get consistent access without the use of a database. ...
    (Pen-Test)
  • Re: Portugal and Azores Ceres question
    ... is the advantage of Access over a flat file database? ... Why risk the typos and the time to query a table like that? ... The report generator in Access is a 2 pass processor. ...
    (rec.collecting.stamps.discuss)