Re: Asynchronous socket operations and threadpool

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

|You don't actually save any work on the host cpus
| by doing things async, you just gain a bit of parallelism on a particular
| user's reqeust.

Agreed. If work is N, then using async will be N+x, where x is the overhead
of thread switches. Naturally, elapsed time can be lower for any given
client by using parallelism.

| The scenario I keep seeing is:
|
| 0 - You get the Socket.BeginRead callback, and get your data. You're now
on
| an IOCP thread.
| 1 - Perform an async operation (say, lookup MX or SRV records in the DNS).
| Pass in a delegate for the callback.
| 2 - While that operation is under way, your IOCP thread keeps going doing
| whatever it can do.
| 3 - Eventually the IOCP thread hits a WaitHandle and has to sync up with
the
| async operation you kicked off.

I thought we are talking about a pure async server? This is more like a
thread per connection server because your blocking on a Wait. In a full
async server, you would not block at all (or for very short times). In
Socket.EndRead, you would get data, update state, and Begin your next async
operation, and on down the line like walking a "virtual" task list
controlled by state. Not pretty to code (I totally agree), but if you need
huge number of connections (i.e. more then ~1500) then maybe is the only
way. So you should not have threads building up because they are blocking.
So most the time, your system will be waiting on BeginReceives and
EndReceives to fire. AFAIK, IOCP thread is not used to wait on the hardware
interrupt. Once the hw fills the read (if ever), that is when the IOCP
thread is invoked to handle the callback.

| At the end of the day, I've found it easier to just do everything
| synchronously once you hit your IOCP callback. This makes for simpler
code,
| easier debugging, far fewer context switches, and (hopefully you won't
need
| to) far, far easier crashdump analysis.

I might flip that around. Use a thread per connection for the client
request, then use async for things that could be done in parallel (i.e. dns,
db, etc). Things like Concurrency Runtime (CCR) from MS will help here for
coordination.

| Just to complicate things, here are some of the other architectures that
| I've tried:
| 1 - As soon as I get a valid data chunk off a socket, I stick it into a
| queue for later processing, then put the socket back into BeginRead mode.
| Use a pool of worker threads [usually a custom threadpool, as too many
other
| things steal thrads from the .Net Threadpool] to pull data off the Queue
and
| process things. This approach seemed like the best canidate for a while,
but
| thread context switching absolutly destroys the performance. There are all
| sorts of issues that also arise, such as how many worker threads to use,
how
| to manage thread affinity on multi-proc systems, how to restart threads
that
| get hung, etc. It turns out that on large, high-availability production
| system this is all very difficult.

Why would you need to worry about thread affinity? Anytime you make an
async call, your effectivity going to pay a context switch tax as well
(unless the call is completed sync). So a threadpool blocking on queue
items would not seem to be more overhead compared to async.
This link shows a variation on this type of server (SEDA):
http://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdf Pretty interesting
read. I am doing this kind of server now, and seems to be working well.
Good discussion. Cheers.

--
William Stacey [MVP]



.

Relevant Pages

  • Problem with popen
    ... Under linux x86 I'm running a stress test which opens some tcp server ... applications and then about 70 clients in each their thread. ... The server however is started as an async process using fork and then ... sync and async processes. ...
    (comp.unix.programmer)
  • RE: Need Help with Page can not be displayed form my web service
    ... On the server side, when you make a call with Begin/End pairing from the ... async request - ... >the web service. ...
    (microsoft.public.dotnet.framework.aspnet.webservices)
  • Re: Async Remoting With Callback
    ... >>>side async calls don't really add up to support a lot of transaction. ... Are you suggesting that there is something inherently slow about async remoting that is ... I have a major server project that does that. ... client) is being suspended for the duration of the remote call itself. ...
    (microsoft.public.dotnet.framework.remoting)
  • Re: epoll,threading
    ... converting a threaded program to a pure async model ... The thing is that the synchronization overhead is something you'll have ... multiple processors on an async program is beyond painful. ... server - e.g. adding coordination with another server (virus scan, ...
    (Linux-Kernel)
  • Re: AppDomain Resetting
    ... But I would suggest that you think about some async mechanism to actually ... run your requests if they are taking this long. ... I am running this app on a Windows 2003 server and SQL ... > issue and I have played with the settings at the server and app level on ...
    (microsoft.public.dotnet.framework.aspnet)