Re: Design issue with WinSock/GetQueuedCompletionStatus

Thanks, but that would not work in my situation. The software I'm creating
is an email server. The worker threads should never stop, so just building a
list of client contextes wouldn't work.

I thought about having some kind list of "dead sockets" where I put sockets
for a short while before I delete them, just to be sure that there's no
outstanding IO operation, but I know some users of my software receives
hundreds of connections per second, and than memory usage would be high even
if the list were cleared every once in a while.

Anyway, I think I solved the problem now. Have thought about it for several
days but managed to solve it just a few minutes after I made my post. I
guess it helped tried to explain for someone else.

MW

"Alexander Nickolov" <agnickolov@xxxxxxxx> wrote in message
news:O13b5DqeGHA.3572@xxxxxxxxxxxxxxxxxxxxxxx
A bit of wisdom - avoid multi-threaded initialization and shutdown
if possible. In your case - don't delete the socket right away -
delegate that to a shutdown routine called after all worker threads
are closed.

--
=====================================
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: agnickolov@xxxxxxxx
MVP VC FAQ: http://www.mvps.org/vcfaq
=====================================

"Martin Wan" <martinwan@xxxxxxxxxxx> wrote in message
news:umj$xZpeGHA.2416@xxxxxxxxxxxxxxxxxxxxxxx
I'm trying to use the GetQueuedCompletionStatus function (in C++) with
sockets but I've run into a design problem. I can't figure out how to use
this function without introducing the risk of application crashes. I've
looked at several examples including the ones in the platform SDK but
still don't understand how to implement it in a good way.

The application I've created is a server accepting connections on a few
different TCP/IP ports and then lets the client run different commands.
Really not anything complex.

Background:

I've created a few worker threads that reads message from the IO
completion port using GetQueuedCompletionStatus().

In the software, a TCP/IP socket can be closed for 2 different reasons:

1) The client closes the connection. In this case,
GetQueuedCompletionStatus() will return 0 and GetLastError() will return
ERROR_NETNAME_DELETED. When this happens, the server will close the
socket and remove the per-socket-client-context from memory (using
operator delete).

2) The server closes the connection. For example, the client can issue a
"quit" command by sending the text "quit" on the socket. When this
happens, my software closes the socket and removes the
per-socket-client-context from memory (using operator delete).

Here's a code snippet that tries to describe the basics (this code isn't
complete and doesn't compile and lacks several things. I only included
the parts that has to do with this issue.)


cIOCP = The IO completion port,
pSocket = The per-socket-client-context,
pOverlapped = The overlapped structure.

while (true)
{
int iResult = GetQueuedCompletionStatus(cIOCP, dwSize, pSocket,
pOverlapped, INFINTE);

if (!iResult && GetLastError() == ERROR_NETNAME_DELETED)
{
// the socket has closed from the client side. delete it
pSocket->close();
delete pSocket;
continue;
}

if (pOverlappedPlus->ioType == IORead)
{
if (pOverlappedPlus->wsaBuf->Contains("quit"))
{
// the client has issued the "quit" command and should be
disconnected
pSocket->close();
delete pSocket;
}
}
}


Problem:
Say the client issues the quit command and then closes the connection
immediately. In this case, two threads may run simultaneously and run
delete() on the same object at the same time which will cause a crash.

Example:

1) Client sends quit
2) Server thread 1 receives the "quit" command and starts parsing it
3) The client disconnects.
4) In server thread 1, the "quit" command and runs pSocket->close() and
delete pSocket
5) In server thread 2, the GetQueuedCompl..() returns 0 because the
client has disconnected. So my software runs pSocket->close() and deletes
pSocket.

Step 4 and 5 might happen at pretty much exactly the same time which will
cause a crash.

Question:
Anyone got any tip or information on how this is ment to work?
I've though about using mutexes or critical sections to circumvent the
problem, but can't see how it would be possible.





.

Relevant Pages

  • Re: Socket switch delay
    ... both at the client and at the server (and why ... would you set the send buffer size to zero on a non-overlapped ... One glaring error is your client does ... So when you use a single socket, ...
    (microsoft.public.win32.programmer.networks)
  • Re: Locking on async calls
    ... you must synchronize the entire SendMessage routine as an atomic ... operation to prevent mixed messages from being transmitted to the server. ... You are correct that read and write on the socket do not interfere with each ... If you want to handle multiple client connections from one server object ...
    (microsoft.public.dotnet.general)
  • Re: socket communication: socket doesnt connect
    ... Microsoft MVP, MCSD ... As far as your server code goes, ... accept the listening socket. ... Client client = new Client; ...
    (microsoft.public.vc.language)
  • Re: TCP server stop receiving new connections
    ... reset the event mask of your listening socket each time you ... I have a strange problem in my class library used by all our client ... server applications. ... incomming connections, but keeps current connections. ...
    (microsoft.public.win32.programmer.networks)
  • Re: GetQueuedCompletionStatus() never returns
    ... The "server" socket needed to be associated with the IOCP. ... > My worker threads do something like this: ...
    (microsoft.public.win32.programmer.networks)