Re: STL Slow - VS2005

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

I do have to see we are having trouble finding any test scenario that runs
significantly on par.


"Jerry Coffin" wrote:

In article <F15505AE-1D5A-4DAF-A294-FE2CF4F2B3EC@xxxxxxxxxxxxx>,
WXS@xxxxxxxxxxxxxxxxxxxxxxxxx says...
Here is the file that was tested with vs2005 stl and vs2005 w/stlport 5.0

The first and most crucial question to ask of any benchmark is how
accurately this benchmark represents real-world workloads. Unless the
benchmark is quite close to real-world usage, it's meaningless -- and
even if it's fairly close, it can still be entirely meaningless (if the
differences between "fairly close" and "real world" make a big
difference in performance).

[ ... ]

string s ;
for ( int i = 0 ; i < 100000000; ++ i ) {
s += " " ;

People have done quite a few studies on string lengths as they're
actually used. Every time I've seen such a study, it's concluded that
most strings are quite short -- most people find that over 90% of
strings are 80 characters or less, and the frequency of usage drops of
quickly with length -- even 1000 characters is extremely rare. A string
of 100 megabytes is so rare that your measurement means nothing about
real use.

[ ... ]

string s("qyweyuewunfkHBUKGYUGL ,wehbYGUW^( @T@H!BALWD:h^&@#*@(#:
JKHWJ:CND");
for (int i = 0 ; i < 10000000; ++i) {
s.find( "unfkHBUKGY" ) ;
s.find( "W^( @T@H!B" ) ;
s.find( "J:CND" ) ;

Likewise, in the real world, most strings consist primarily (if not
exclusively) of a-z, A-Z, space, and a FEW punctuation characters. I
haven't checked, but my immediate guess is that a significant difference
in string searching speed likely results from using a Boyer-Moore (or
BMH) search algorithm. Restricting the alphabet in use also
significantly reduces the advantage gained from such an algorithm.

That's not to say that such an advanced algorithm can't be useful -- but
my immediate guess is that the strings chosen above significantly
exaggerate the advantage likely to be gained in most real use.

string s("qyweyuewunfkHBUKGYUGL ,wehbYGUW^( @T@H!BALWD:h^&@#*@(#:
JKHWJ:CND");
[ ... ]
printf("Assignment, search, replace 10 million times: %.0f ms\n", msecs);

The comments about searching apply here as well.

[ ... ]

string s("qyweyuewunfkHBUKGYUGL ,wehbYGUW^( @T@H!BALWD:h^&@#*@(#:
JKHWJ:CND");
[ ... ]
printf("Pass and return by value function called 10 million times: %.0f
ms\n", msecs);

I can't imagine anybody passing strings by value except by accident.
Virtually every book on C++ teaches even rank beginners to pass class-
objects by ref-to-const by about chapter 2. This benchmark seems to
apply only to thoroughly inept code.

string s("qyweyuewunfkHBUKGYUGL ,wehbYGUW^( @T@H!BALWD:h^&@#*@(#:
JKHWJ:CND");
[ ... ]
printf("Pass by reference return by value function called 10 million
times: %.0f ms\n", msecs);

The first comments on string length apply here as well, though obviously
to a somewhat lesser degree -- this string is somewhat longer than
usual, but at least not nearly so outrageously so.

string s("1234567890");
[ ... ]
printf("Short String - Pass and return by value function called 10 million
times: %.0f ms\n", msecs);

Passing by value again? If I didn't know better, I'd guess you carefully
avoided the two (by far) most common situations: 1) pass a short string
by reference and return a short string by value, 2) pass a short string
by reference, modify a string whose reference was passed.

Of course, the second of these doesn't really measure much about the
string implementation at all -- and if the compiler includes named
and/or anonymous return value optimization, the first probably doesn't
either. To make a long story short, this entire line of testing only
means something to 1) people who have no clue of how to use C++, or 2)
are stuck using old compilers without any return value optimization
capability.

Given STLPort's target of supporting nearly all compilers, the latter is
probably meaningful for the library. Unless I'm mistaken, VS 2005 does
include RVO, so it means little in this context though.

Ultimately, I can't see anything here that seems to indicate much of
anything about whether (for example) I'd get better or worse performance
in my programs if I used STLPort's string class instead of Dinkum's. It
might happen, but these tests just don't show anything meaningful one
way or the other.

--
Later,
Jerry.

The universe is a figment of its own imagination.

.

Relevant Pages

  • Re: STL Slow - VS2005
    ... The first and most crucial question to ask of any benchmark is how ... accurately this benchmark represents real-world workloads. ... it can still be entirely meaningless (if the ... People have done quite a few studies on string lengths as they're ...
    (microsoft.public.vc.stl)
  • Re: fastest count of instances in string?
    ... > and call that in the loop. ... then you should make all the benchmark ... but I can't figure how putting the tr// in a sub ... splitting the string and looking through the array). ...
    (comp.lang.perl.misc)
  • Re: QC Report - FastMM4 is Slow!!
    ... I do not see that piece of code as an example of string handling. ... I think that our benchmark suite should have such benchmarks and I think it ... most common situations. ...
    (borland.public.delphi.language.basm)
  • Re: STL Slow - VS2005
    ... tests and only glanced at the test scenario, ... The first and most crucial question to ask of any benchmark is how ... it can still be entirely meaningless (if the ... People have done quite a few studies on string lengths as they're ...
    (microsoft.public.vc.stl)
  • Re: STL Slow - VS2005
    ... accurately this benchmark represents real-world workloads. ... People have done quite a few studies on string lengths as they're ... quickly with length -- even 1000 characters is extremely rare. ... by reference, modify a string whose reference was passed. ...
    (microsoft.public.vc.stl)