Re: Can anyone tell me of any optimations I could do to this to make it faster?

Tech-Archive recommends: Speed Up your PC by fixing your registry

Extremest@xxxxxxxxxxxxx wrote:

I know there are ways to make this a lot faster. Any
newsreader does this in seconds. I don't know how they do
it and I am very new to c#. If anyone knows a faster way
please let me know. All I am doing is quering the db for
all the headers for a certain group and then going through
them to find all the parts of each post. I only want ones
that are complete. Meaning all segments for that one file
posted are there.

Rule number one of optimizing code is going back to your algorithm and
see if that's optimal. Often you can spot 'hotspots' in an algorithm
quite easily, for example if you have a part of the algorithm which has
to be performed a lot of times. It's then best to invent a NEW
algorithm which does things more efficient. Often this requires to
start from scratch and do things completely different.

After you've optimized your algorithm, modify your code so it matches
the new algorithm.

Rule number two is measuring, with software performance measuring this
means: profiling. Download a .NET profiler and measure your code. Only
then you'll KNOW which parts are slow and which parts arent. If you
don't measure / profile your code, you will never be able to optimize a
slow piece of code, as chances are you'll have to guess which parts are
slow and will then optimize things which aren't slow or not significant
in the whole process.

Rule number three is that you have to avoid micro-optimizations. This
means that you always, in all cases, have to start from rule number 1,
and then do rule number 2. Micro-optimizations are what is done in this
thread, no offence to the people who helped you out as all they have is
your code. When you're doing micro-optimizations you look at the code
and guess which parts are slow, and then try to change them with what
you think are faster constructs.

Often this doesn't make a difference or makes things worse. The thing
is: if you have a slow piece of code but it is run once and takes 0.3
seconds to complete and you have a tight loop which takes 0.01 seconds
to complete but is run 10,000 times, which part of the code is
significant for the run of your program? The tight loop might look
fast, but in the end it's the bottleneck, not that piece of code which
takes 0.3 seconds.

Though as I said before, people in this newsgroup only have your code
snippet, so have to fall back on micro-optimizing, as there's no
explanation of the algorithm nor are there design decision motivations
available.

The thing which to me looks really slow is the LIKE predicate in your
query. LIKE is slow, especially when you use wildcards like the way you
do it.

To give you a hint of how an algorithm change can help you
tremendously here (as an example of rule nr. 1): what will your app do
the most: reading or writing? My guess: reading. So it will spend say
90% of its time reading data over and over again and 10% of its time
saving data.

This thus means that you have to optimize for reading, not writing.
This thus means that you have to avoid doing as much operations as
possible when you read data. It should be as firing a select, and
dumping the results on the screen, simplisticly said. So you should
move the processing of what's inside the DB to the WRITER logic. There,
you know what's going to be saved or better: you can analyse it and
retrieve extra information from it when it's written. THIS information
is then also stored in the DB.

When you READ data, you then simply use a couple of JOINs and simple
WHERE predicates (so no LIKE's) to fetch the data you need and you can
completely avoid the processing of data read.

This is an algorithm change, but it will beat any code-optimization
hands down.

Always pre-calculate as much as you can to avoid stalls in often used
code. Process data to get info over and over again? Do it once and
create a lookup table at runtime, saves you processing in all
subsequential reads. Simple, yet very effective.

Good luck :)

FB

--
------------------------------------------------------------------------
Lead developer of LLBLGen Pro, the productive O/R mapper for .NET
LLBLGen Pro website: http://www.llblgen.com
My .NET blog: http://weblogs.asp.net/fbouma
Microsoft MVP (C#)
------------------------------------------------------------------------
.

Relevant Pages

  • Re: Patents Unleashed and the future of Java Programming
    ... Have you ever seen documented an algorithm for determining the number ... However, I disqualify "reading coworkers code", "looking at ... Even the most anal retentive documentation efforts (I've worked on ...
    (comp.lang.java.programmer)
  • Re: Proving or disproving new search algorithm is better than binary search.
    ... If I were reading it, I would expect to see several ... search algorithm to find a specific truck with specific cargo using ... you will have to invent a problem set. ... if you are publishing a paper on a topic where no ...
    (comp.theory)
  • Re: When is the FTP version available?
    ... houghi wrote: ... even better would be to change the algorithm so it stops ... >> reading the file after a match, or when there's no more headers. ...
    (alt.os.linux.suse)
  • Re: [XPOST] A unique number for every "person" - can it be done?
    ... > existence of an algorithm to assign numbers to people have any moral ... > been reading too much sci fi. ...
    (comp.programming)
  • Re: need for JPEG successor?
    ... What you should avoid are the not-so-well-done reference implementations that barely do the absolute minimum, or close to the absolute minimum. ... And that also is part of the problem with the JPEG2000 implementations you find out in the wild - most of them are just optimal to PSNR, and optimize for PSNR - which is just the wrong thing - as you say. ... instead of a retarded algorithm. ... doing subjective tests is only half the job done - how do you write an algorithm that optimizes for this definition? ...
    (comp.compression)