Re: Performance - 0 ticks ???

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

From: Ian Griffiths [C# MVP] (ian-interact-sw_at_nospam.nospam)
Date: 12/01/04 

Date: Wed, 1 Dec 2004 19:29:04 -0000

Of course timing repeated execution of the operation is misleading too...
It's true that it provides a very accurate measure of how quickly the code
runs when you call it lots and lots of times in quick succession. The
problem is that this doesn't necessarily tell you how fast it will run when
used in the real-life context in which it will eventually be used.

The speed at which a particular piece of code executes varies wildly
according to what the CPU has been doing recently. How good a measure
you'll get from repeatedly running the code in a tight loop will depend on
how the code will get called in real life. If that's how it's going to be
used in reality, then this will be a good test. If it's not, then it
probably won't be.

The very first time the CPU runs a particular bit of code, it'll be fairly
slow - it will have to load the code out of main memory, which is pretty
slow. But it you run it immediately thereafter, it will be much much
faster, because it's in the cache now - this speeds things up by orders of
magnitude. And there's also the similar issue of whether the data it works
with is in the cache or not. If your simple looping test works with the
same data each time round, but in real life it's going to have to work with
data that might not yet be in the cache, you're going to get pretty
misleading results.

The problem is that microbenchmarks can be misleading by an order of
magnitude or more. The only way to get meaningful results is to try out a
realistic workload, and see what the impact of individual changes are on
your throughput/latency/whatever performance metric matters most to you.

But given what it sounds like the customer has asked for here, it looks like
they are simply demanding a test that will produce some entirely meaningless
data... 

-- 
Ian Griffiths - http://www.interact-sw.co.uk/iangblog/
DevelopMentor - http://www.develop.com/
"Frank Hileman" wrote:
> By the way, Jon Skeet is correct: timing a single short operation produces 
> meaningless data. You must loop in an operation that takes a few 10s of 
> milliseconds, minimum, to obtain anything useful, and even that is fairly 
> meaningless unless your CPU is running a single process (impossible). So 
> plan on looping a thousand times for your little operation, and then loop 
> that loop 10 or 20 times, getting the elapsed time each time, then look at 
> the average of those times, possibly excluding the first loop.
>
> At least the performance counter does produce accurate data. TimeSpan is 
> not accurate.
>
> "Tamir Khason" wrote:
>> Follow the code I have to check the performance
>> double a = r.Next(9)*1e307;
>>
>>
>> DateTime d = DateTime.Now;
>>
>> double b = Math.Pow(a,1/(double)100);
>>
>> TimeSpan tm = DateTime.Now - d;
>>
>> Console.WriteLine("{0}---{1}",b,tm.Ticks);
>>
>> The return value is 0 Ticks - IT CAN NOT BE !!! What is the problem?

Relevant Pages

  • Re: unique numbers using srand( ) and rand( ) functions in C++
    ... > I'd suggest that you acquaint yourself with current standard headers ... > a total of 50 groups, which is in itself extremely misleading. ... > of numbers up to a point the loop determines. ... This must be done regardless of any ...
    (comp.lang.cpp)
  • Re: Code Comprehension
    ... contrary, they may be misleading. ... that, because you did not eliminate the "for" loop, which is ... that a 0 by itself might have an integral type with a larger ... However, with any loop that has an empty body, I ...
    (comp.programming)
  • Re: [PHP] array problems
    ... $city is a string and $content_array is an ... array. ... It's meaningless to compare them. ... > inside the while loop $content_array is available to me. ...
    (php.general)
  • Re: Pretty cool wind averag map for US
    ... meaningless? ... averages are very misleading. ... Take an area like Amarillo, ... wind blows 20mph all the time. ...
    (rec.windsurfing)
  • Re: strange results using m//g in while loop...
    ... though it never seems to enter the loop: ... so \G is meaningless in list context and you regex fails and the loop ...
    (comp.lang.perl.misc)