Design issue with WinSock/GetQueuedCompletionStatus

Tech-Archive recommends: Fix windows errors by optimizing your registry



I'm trying to use the GetQueuedCompletionStatus function (in C++) with
sockets but I've run into a design problem. I can't figure out how to use
this function without introducing the risk of application crashes. I've
looked at several examples including the ones in the platform SDK but still
don't understand how to implement it in a good way.

The application I've created is a server accepting connections on a few
different TCP/IP ports and then lets the client run different commands.
Really not anything complex.

Background:

I've created a few worker threads that reads message from the IO completion
port using GetQueuedCompletionStatus().

In the software, a TCP/IP socket can be closed for 2 different reasons:

1) The client closes the connection. In this case,
GetQueuedCompletionStatus() will return 0 and GetLastError() will return
ERROR_NETNAME_DELETED. When this happens, the server will close the socket
and remove the per-socket-client-context from memory (using operator
delete).

2) The server closes the connection. For example, the client can issue a
"quit" command by sending the text "quit" on the socket. When this happens,
my software closes the socket and removes the per-socket-client-context from
memory (using operator delete).

Here's a code snippet that tries to describe the basics (this code isn't
complete and doesn't compile and lacks several things. I only included the
parts that has to do with this issue.)


cIOCP = The IO completion port,
pSocket = The per-socket-client-context,
pOverlapped = The overlapped structure.

while (true)
{
int iResult = GetQueuedCompletionStatus(cIOCP, dwSize, pSocket,
pOverlapped, INFINTE);

if (!iResult && GetLastError() == ERROR_NETNAME_DELETED)
{
// the socket has closed from the client side. delete it
pSocket->close();
delete pSocket;
continue;
}

if (pOverlappedPlus->ioType == IORead)
{
if (pOverlappedPlus->wsaBuf->Contains("quit"))
{
// the client has issued the "quit" command and should be
disconnected
pSocket->close();
delete pSocket;
}
}
}


Problem:
Say the client issues the quit command and then closes the connection
immediately. In this case, two threads may run simultaneously and run
delete() on the same object at the same time which will cause a crash.

Example:

1) Client sends quit
2) Server thread 1 receives the "quit" command and starts parsing it
3) The client disconnects.
4) In server thread 1, the "quit" command and runs pSocket->close() and
delete pSocket
5) In server thread 2, the GetQueuedCompl..() returns 0 because the client
has disconnected. So my software runs pSocket->close() and deletes pSocket.

Step 4 and 5 might happen at pretty much exactly the same time which will
cause a crash.

Question:
Anyone got any tip or information on how this is ment to work?
I've though about using mutexes or critical sections to circumvent the
problem, but can't see how it would be possible.


.



Relevant Pages

  • [PATCH 0/5] [RFC] AF_RXRPC socket family implementation [try #3]
    ... These patches together supply secure client-side RxRPC connectivity as a Linux ... kernel socket family. ... presentation side is left to the client. ... Each connection goes to a particular "service". ...
    (Linux-Kernel)
  • [PATCH 0/5] [RFC] AF_RXRPC socket family implementation
    ... These patches together supply secure client-side RxRPC connectivity as a Linux ... Make it possible for the client socket to be used to go to more than one ... Each connection goes to a particular "service". ...
    (Linux-Kernel)
  • [PATCH 0/5] [RFC] AF_RXRPC socket family implementation [try #2]
    ... These patches together supply secure client-side RxRPC connectivity as a Linux ... Make it possible for the client socket to be used to go to more than one ... Each connection goes to a particular "service". ...
    (Linux-Kernel)
  • [PATCH 00/16] AF_RXRPC socket family and AFS rewrite [try #4]
    ... The first of these patches together provide secure client-side RxRPC ... connectivity as a Linux kernel socket family. ... Make certain parameters (such as connection timeouts) userspace ... the connectaddress of a client socket by making use of msg_name in the ...
    (Linux-Kernel)
  • [PATCH 00/16] AF_RXRPC socket family and AFS rewrite [try #3]
    ... The first of these patches together provide secure client-side RxRPC ... connectivity as a Linux kernel socket family. ... Make certain parameters (such as connection timeouts) userspace ... the connectaddress of a client socket by making use of msg_name in the ...
    (Linux-Kernel)