Re: Socket switch delay

From: Alun Jones [MSFT] (alunj_at_online.microsoft.com)
Date: 08/24/04 

Date: Mon, 23 Aug 2004 17:07:32 -0700

"mil" <mil@discussions.microsoft.com> wrote in message
news:1EF5AB37-064F-441F-976D-0D39029DE146@microsoft.com...
> I was referring to the client sending data on this socket while the server
> was also sending data in the same time. Which seems to be a big no no.

What you're talking about is commonly referred to as "asynchronous
operation" - each turn does not need to wait its turn before sending. TCP
_definitely_ supports asynchronous operation, so it would be impossible for
sockets to provide TCP support without also supporting asynchronous
operation.

What _is_ a big no-no is when two threads vie for sending on the same
socket. A basic tenet of TCP is that the order of data going into the
stream is the same as that coming out, and two threads on the same machine
both trying to send would get in the way of that tenet.

> My client already uses non-blocking sockets, with the combination of the
> select statement, so it will never "lock" while waiting to send or
receive.
>
> The server uses blocking sockets just because I am also using Overlapped
IO
> structures to send the packets. Thus I don't need to care if it locks or
not
> since the Overlapped IO will never block me.

So, if you're not blockable, why are you being blocked in a receive? If, as
you say, the overlapped I/O means you aren't blockable, then set the socket
to be non-blocking, so that your socket doesn't block in receive.

> The reason I am using a 0 bytes send buffer in my socket (i.e. I setup the
> socket not to use any buffering, which actually works on XP but not on
> Windows 2000 in the non-blocking sockets) is because all the communication
is
> done in packets already. Each packet is 32KB max and I can send one or
more
> of those in one go. The server and client, save all the data they want to
> send in these packets, those get compressed (if possible/necessary) and
> encrypted (through the MS Crypto API) and then I execute a WSASend with
> multiple buffers. So I don't need the protocol to buffer the data for a
> second time, since I let it use my buffers for as long as it likes.

Trying to do "packets" on TCP is usually a bad idea. TCP is a stream, it's
going to ignore your packets. If it gets half a chance, it will chop your
packets up, and assemble them so that parts of two of your packets go out in
the same IP packet on the wire. Don't think of TCP as handling packets - it
doesn't.

As an example of why this might be a bad idea, note that there are cards
that offload TCP handling from the system to the card, handling TCP
buffering internally - setting the send buffer size to zero will mean that
your program suffers from worse performance on such a card. [A search on
"Winsock Direct" can give you more information on this than you ever wanted
to know].

> The receiver will always get a 256 byte header that describes what "is
> coming" from the other end and will load the number of blocks with their
> proper size, decrypt/decompress and pass them to the rest of the app.

Yes, this is a common cause of Nagle / delayed ACK interactions - "we send a
header, followed by data, and wait for a response" - you've got the
send/send/recv pattern encoded in text right there.

As long as you hold onto the header and the data, and send the two in one
call, you'll get better performance from that part of your code.

> After our conversations about the delay, I changed my client code so it
uses
> 2 connections again but now:
>
> The first connection is used to send and receive packets "synchronously",
by
> sending the data, then blocking on an "Event", which another thread
receives
> the reply and "Sets". When the client wants to send a number of requests
> without conflicting with server replies (i.e. wants delayed replies), it
> simply tells to the server to use the second socket (channel) to send the
> replies. Another thread handles the receive again from this second channel
> and then the client (after it has finished with all the send-requests)
blocks
> on "multiple events".

This sounds overly complicated. Simplify it. Your socket already contains
two asyncronous channels - one in each direction, inbound and outbound. To
interact well with Nagle, you can either be completely asynchronous, and
send and receive data willy-nilly, or do lock-step synchronous sends and
receives. What you must do, though, is to group all related data into a
single call to send() or WSASend().

> Since the second socket is used to send file data, or feedback messages
> generated by the server, I don't care about the 200ms delay, actually in
this
> case it really helps when there are many small packets generated by the
> "server feedback messages", for the reasons you explained to be already.

If you're sending file data, you will largely overcome any problems with
Nagle / delayed ACK by keeping the stream full of data - you'll always have
an MTU's worth to send, so Nagle won't slow you down, and you'll keep
triggering the ACKs by having more than two segments' worth of data to
acknowledge.

> The whole thing now works so nicely that you almost cannot tell the
> difference when the server is running on a dual PIII or a dual Opteron
248.
> 0% CPU usage when idle, instant responses on requests (and that includes a
> round trip to the Access through OLEDB in the server side).

I still feel like you've got something that's a little more baroque than
necessary. Still, I could be wrong, so if it ain't baroque, don't fix it.

Alun.
~~~~

Relevant Pages

  • Re: Problem with writing fast UDP server
    ... UDP packets per second. ... socket and threads. ... I wrote a simple case test: client and server. ... The maximum theoretical limit is 14,880 frames per ...
    (comp.lang.python)
  • Re: Http server implementation for Windows Media Server
    ... I'm assuming you are using a raw TCP ... Dump that idea and use a regular TCP socket instead. ... the packets aren't ACKed because ... since my Http server simulates for him a Windows Media ...
    (microsoft.public.win32.programmer.networks)
  • Re: network programming: how does s.accept() work?
    ... The articles and books I've read all claim that the server ... port 5053 is a 'listening' port only. ... the server creates a new socket for communication between the client ... on the network, and in the RFCs which define the TCP protocol (UDP too, but ...
    (comp.lang.python)
  • Re: Detecting when a socket has been closed.
    ... shutdown the server closing the socket the client still thinks the socket is open. ... First send will not throw exception, because there is no way for TCP stack to know that other end has actually closed its receiving side. ... It takes some time for packet to reach other side and return information that server side is gone. ... in case other end has closed socket and network is still operating after first send TCP RST segment will be returned to TCP stack notifying it connection has gone down. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Socket switch delay
    ... The server uses blocking sockets just because I am also using Overlapped IO ... structures to send the packets. ... The reason I am using a 0 bytes send buffer in my socket (i.e. ...
    (microsoft.public.win32.programmer.networks)