Re: Asynchronous socket operations and threadpool

"Yifan Li" <nomail@xxxxxxxxxxxx> wrote

I have a general question regarding the best practice for using
asynchronous
socket operations and threadpool.

Async sockets don't really interact with the .Net threadpool, so your
question isn't quite right. Async sockets really leverage the I/O Completion
Port Thread Pool, which is a very different beast.

It often happens that I need to perform lenghty operations after
asynchronous receive/send/accept. These operations themselves are often
composed of multiple atom operations which can be completed
asynchronously.
For example, upon accepting a connection I might want to lookup the remote
host using dns, and then send/receive some data. These things must be
completed in order (e.g. have to wait for dns to finish before sending).

Alright. I've done very similar things in the past when building Coversant's
SoapBox server, which is a highly scalable XMPP server written all in .Net.
We're on the same page so far.
Lengthy operations generally include: ADO.NET Operations, DNS Operations,
Encryption, Policy Applications, and so on down the line.

I understand that the callbacks are queued and executed on a threadpool
thread, which is limited in quantity.

This isn't true, as I mentioned above. The threads your callbacks are on are
IOCP threads, not .NET threadpool threads. There are 1000 IOCP threads
available for use by default, not the 25 per processor that the normal
thread pool has.

I believe that this will become a bottle neck of my application if the
subsequent actions are done in a synchronous fasion. Should I simply
increase the number of thread pool threads (which is easy) or should I
make everything asynchronous?

It will, in all likleyhood, be your limitation no matter what you do. On the
positive side you have 1000 threads to play with, not 25, so the limitation
is prety high. On the downside, when you do finally hit the limit it's a
difficult thing to get past. On the very downside, stepping through 1000
threads in WinDbg using Son of Strike against a crashdump really sucks.

I can say it's easier and less bug prone to get data from a call to an async
socket, then process all that data synchronously. Otherwise you're dealing
with quite a bit more complexity to get all the thread events to sync up the
right way, and the failure cases get... crazy.

Also, if you have sy 600 IOCP threads processing at once, there's really no
way to do other async operations - you can't use the .Net thread pool, as it
doesn't have enough threads. You certainly don't want to spin up hundreds of
your own threads, so pragmatically it's normally best to do things
synchronously within the IOCP callback.

What will be the complications if I simply increase the number of
threadpool threads?

Long before you run into IOCP threadpool issues, you're going to run into
heap fragmentation issues. Each time you put a socket into an async mode
(BeginRead, is the cannonical culprit here), your receive buffer get's
pinned in the heap for the handoff to Win32 land. This pinning will create
lots of holes in the heap, and cause you to run out of memory long before
you would otherwise experct. The 2.0 GC algorithms have improved this, but
it's still likley to be the problem you run into first.

This is discussed in detail at:
https://blogs.msdn.com/yunjin/archive/2004/01/27/63642.aspx

The CLR 2.0 improvements are discussed here:
http://blogs.msdn.com/maoni/archive/2005/10/03/476750.aspx

--
Chris Mullins


.

Relevant Pages

  • Re: runaway thread count and asynchronous sockets
    ... explicitly spawn in my server a normal priority. ... threads spawned by the CLR to handle the async socket communication. ... >> thread you spin off so high thread counts in and of themselves are not always ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Socket returns no data
    ... When using async, normally these issues can be traced back to small coding ... > I have run into a problem which may be a bug in the socket class of .net. ... > data flow and send packets one at a time. ... > it should have put the data in is still filled with zeros (nulls). ...
    (microsoft.public.dotnet.framework)
  • Re: 5 proposals to enhance network socket classes in .NET
    ... Socket uses IOCompletionPorts on platforms that support it. ... platforms, it uses Overlapped I/O. ... we dont simulate async by creating a background thread and doing ... the async methods are signifigantly better. ...
    (microsoft.public.dotnet.framework)
  • runaway thread count and asynchronous sockets
    ... I've got a server that handles a relatively high number of concurrent ... applications establish socket connections with the server. ... they're being spawned by the CLR to handle the high volume of async ...
    (microsoft.public.dotnet.languages.csharp)