Re: Custom Thread Pool (by Mr. Jon Skeet) Enhancement
- From: Jon Skeet [C# MVP] <skeet@xxxxxxxxx>
- Date: Wed, 21 Sep 2005 01:57:13 +0100
Kieran Benton <kieranbenton@xxxxxxxxxxx> wrote:
> I'm currently in the tail end process of developing a high scalability
> server for my employer. Essentially it receives short socket based
> connections with an ASCII message, parses that message, does some
> processing and then sends out a string reply on the same connection.
>
> I'm using the asynchrounous IO completion port based socket methods in
> .NET 2.0 to handle the comms side of things (giving extremely good
> performance) and the venerable Mr. Skeet's custom threadpool to execute
> the processing portion of the system. My problem is that not all of the
> messages are created equal. Some simply do a very quick query from a DB
> (or even a cache), whereas others do a credit card authorisation or
> some longer SQL work.
>
> Because of this I've tried to seperate my messages into two classes,
> short running and long running, running them on two seperate
> threadpools. Its become very hard to manage how many threads I allocate
> to each - as all messages involve some I/O blocking I want to avoid
> starving the pools as much as possible to keep latency low.
Right. Could you not use asychronous IO again, and keep the thread pool
for actual processing, so that any thread which is in the pool is
either running or available, not blocking in a useless way? It's no
doubt relatively tricky to write the code that way, but it sounds like
the way to get the maximum performance out.
> Based on this article:
> http://blogs.msdn.com/cbrumme/archive/2004/02/21/77595.aspx (Threading
> and Synchronization, near the top - he explains the concept much better
> than I ever could), I've been thinking about expanding Jon's ThreadPool
> to automatically determine its own maximum number of threads to keep
> CPU usage high (by increasing the number of threads) whilst not choking
> the machine to death with too many context switches. This is apparently
> a big simplification of what MS SQL Server does internally, and I'd
> like to replace my two pools with jsut one of this nature.
Have you tried just using a relatively large pool, and measuring how
much time is actually spent context switching? Do you definitely have a
problem?
> Essentially I'd just like anyone's comments (especially yours Jon!) as
> to whether this is worthwhile, or how to go about it - I've got some
> ideas of my own:
> 1. Check on completion of a work item the CPU usage and decide whether
> to create a new thread, keep the thread count the same, suspend the
> thread?
> 2. Add a timer (100ms?) that checks the CPU periodically and sets the
> max and min thread values? How does CPU usage work with multiple CPUs?
>
> Its getting to be a bit of a pain as I have to tune the system as it is
> to the machine it runs on and the type of load it encounters!
You certainly could take either of those approaches, but I don't know
whether they'd really do what you want them to. I'd definitely try
profiling with a few different sizes of threadpool first.
--
Jon Skeet - <skeet@xxxxxxxxx>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
.
- References:
- Custom Thread Pool (by Mr. Jon Skeet) Enhancement
- From: Kieran Benton
- Custom Thread Pool (by Mr. Jon Skeet) Enhancement
- Prev by Date: Re: Need help with regular expression.
- Next by Date: Re: Are gotos truly evil?
- Previous by thread: Custom Thread Pool (by Mr. Jon Skeet) Enhancement
- Next by thread: Re: Custom Thread Pool (by Mr. Jon Skeet) Enhancement
- Index(es):
Relevant Pages
|
