Re: Parsing large amounts of data (200,000 entries) with XML?
From: jamie (no_at_spam.com)
Date: 03/13/04
- Next message: Randy Birch: "Re: runtime 53 stating dll not found - BUT: it really is where its supposed to be"
- Previous message: jamie: "Re: Parsing large amounts of data (200,000 entries) with XML?"
- In reply to: Jim Carlock: "Re: Parsing large amounts of data (200,000 entries) with XML?"
- Next in thread: Tony Proctor: "Re: Parsing large amounts of data (200,000 entries) with XML?"
- Messages sorted by: [ date ] [ thread ]
Date: Sat, 13 Mar 2004 10:41:35 -0500
Jim Carlock wrote:
> I'm thinking the once every 1000 to 5000 should be bumped
> up, to maybe 100000 or 500000. But it's all going to depend
> upon the actual amount of processing that is going on.
>
> Simple queries are extraordinarily quick and I wouldn't place
> any kind of other processing in a simple query loop.
>
> I've created reports in the past using Microsoft Access, that
> have read millions of records and performed a variety of
> calculations on the records, and in turn the month end reports
> were 1000 pages in print and were AR statements going out
> to Home Depot, Ace Hardware, Lowes and a variety of other
> smaller hardware stores that bought paint.
>
> I won't put down XML, and I'm sure it can be made to be
> processed almost as quickly as Access can process things, so
> with the current speed of processors, and even dating back to
> 100MHz processors... I think it's safe to bump the numbers
> up into the 100 thousands.
>
> Also, when dealing with data, it's important to start thinking in
> terms of SQL and in terms of Tables, Recordsets and Records.
> I didn't add queries into that list because queries are one of two
> forms that act upon the items I listed. Queries are used to either
> modify sets of data or the presentation of sets of data.
>
> It makes things a little easier to understand. You get a table of
> records, or you get a particular record, or you update a set of
> records or a subset of the table.
>
> XML is going to be a bit slower than reading from a linked list
> of records, because on every read of every record, there is
> processing of the XML tags. This extra processing is what makes
> it slower. Because in effect, you don't deal with tables or sets
> of data anymore, but you deal with elements of data, each element
> being represented by whats in a set of tags.
>
> Hope that helps.
>
Thanks Kim & Rick,
My problem was how I designed the XML layout, Joseph has pointed out a
much more efficient structure for me to be using
Regards,
Jamie
- Next message: Randy Birch: "Re: runtime 53 stating dll not found - BUT: it really is where its supposed to be"
- Previous message: jamie: "Re: Parsing large amounts of data (200,000 entries) with XML?"
- In reply to: Jim Carlock: "Re: Parsing large amounts of data (200,000 entries) with XML?"
- Next in thread: Tony Proctor: "Re: Parsing large amounts of data (200,000 entries) with XML?"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
