Re: WSARecv & IOCP...when exactly is the notification sent???

Thanks Vladimir & m, your posts were really helpful!


"m" wrote:

The part that is confusing you is that you are forgetting about protocol
processing in the network stack. When a packet comes in off the wire (or
thru the air), it is processed by each protocol driver bound to that
interface. Each protocol will discard any packets that it doesn't
understand (because they are part of another protocol etc. - which it how it
is possible to run NetBIOS & IP & IPX etc. on the same network). Look at
NDIS in the DDK if you want more info on what a protocol driver does and how
it interfaces to the hardware.



In the specific case of TCP / IP sockets on Windows, Microsoft has provided
an NDIS protocol driver, and a bunch of other drivers and interfaces, to
process the IP and TCP protocols in kernel mode. What you get out of
WSARecv is access to the TCP stream as it is assembled by the protocol
driver. This is an oversimplification, but is a good way of thinking about
the process.



When deciding how to interface with the Winsock API, remember this:



- A call to WSARecv returns when some new piece of the TCP stream
becomes ready for the application

- The size of the data returned by a call to WSARecv is arbitrarily
determined by the protocol processing and ranges from 1 byte to the full
size of the supplied buffer

- There is no correlation between the size of the data returned by
WSARecv and the size of buffered supplied to WSASend

- There is no direct correlation between the arrival of a packet and
a return from WSARecv. A call to WSARecv could be satisfied from data that
has already arrived and been assembled; a single packet could provide data
for multiple WSARecv's, or a single WSARecv could return data from multiple
packets.

- The protocol processing uses buffer space in kernel mode (usually
non-paged memory) to perform the protocol processing. If you have a pending
call to WSARecv, then it will use the buffer that you supply directly; if
there are no pending calls to WSARecv, then the driver will allocate its own
buffer and then later copy the data to the one you supply in WSARecv. As
the driver's buffer fills, it decreases the TCP window size and the data
transmission rate slows (again an oversimplification, but you get the
point - you can lookup the details in the RFC)



So . all of this is to say that, unless you have an extremely large number
of connections (10,000+), pending a 4KB read to each will improve
performance (by eliminating a memory copy & preventing network stalls) for
most data rates on most hardware.



I recommend posting multiple buffers (with multiple calls to WSARecv) rather
than posting larger buffers if this scheme doesn't meet performance
requirements, even though is uses more system resources, because it allows
the driver to continue to use a user buffer to assemble the TCP stream while
the results from a WSARecv call are being processed by the application. The
complication with this design is that, while the buffers are filled FIFO as
the driver 'sees' them, if the IO is issued from multiple threads (as in
most IOCP based designs), careful sync is required to ensure that the
buffers are put back together in the same order as they were filled by the
driver.



"George" <George@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:79CCB524-7F58-43E4-9FF0-239D189F2BCA@xxxxxxxxxxxxxxxx
my understanding is that tcp data arrives in packets with MTU around 1500
bytes. Here is what i'm confused at: when the remote socket calls send()
with
4KB of data then would zero-byte recv return as soon as the first packet
(transmission unit) is received? Would it be a wise-choice to use small
buffers 4K size of use with zero-byte recvs?

For first one, you mentioned that wsarecv with return with whatever bytes
there are available. So it is exactly similar to zero-byte recv with only
advantage being that the data is copied to provided buffer?



"Vladimir Petter [MSFT]" wrote:

- The operation will be completed with whatever number of bytes are
available. It would not wait for all bytes to arrive.
- Operation will be completed as soon as there is a single byte
available.
Vladimir




.

Relevant Pages

  • Re: WSARecv & IOCP...when exactly is the notification sent???
    ... The part that is confusing you is that you are forgetting about protocol ... NDIS in the DDK if you want more info on what a protocol driver does and how ... size of the supplied buffer ... WSARecv and the size of buffered supplied to WSASend ...
    (microsoft.public.win32.programmer.networks)
  • My first driver
    ... The protocol I have to work with limits me to the number of bytes the dongle ... This lets the dongle just pass through the data. ... I guess what it comes down to is can I use the driver to put data for this ... this "other" buffer and use it for it's intended purpose. ...
    (microsoft.public.development.device.drivers)
  • Re: Difference between synchronous and asynchronous operation/calls (NDISPROT)
    ... The main problem with synchronous operation/calls seems to be the lack of buffers for the driver to store data into. ... Each "read call" supplies the driver with a buffer and optionally a completion routine so that the driver can inform the application when the requested operation is/was done. ... However it seems the current windows/driver design does some copies as well? ...
    (microsoft.public.development.device.drivers)
  • Re: Degradation of TCP connection
    ... Gigabit ethernet. ... D card's data buffer can only hold about 64K samples worth of data ... link you posted is for an older version of VxWorks that used a BSD- ... but a bug in the ethernet driver. ...
    (comp.os.vxworks)
  • Re: PCI bus-master and large contiguous memory buffers
    ... As soon as device reaches the end of the buffer ... Sure, I am developing both PCI adapter and device driver, so, it is ... not afford reinitializing DMA on my device after every transfer. ... x86 CPU memory management structures I never tried to dig into Windows ...
    (microsoft.public.development.device.drivers)