Re: Deaf CAsyncSocket on Windows Service.

See below...
On Wed, 21 Jan 2009 10:27:20 +0800, "Bill" <<don't want more spam>> wrote:

I never once said the word 'packet'.

There are a number of bytes of data to be read and I can't find out how many
bytes that is except to read them out. To read them out I need a buffer to
put them into and I don't know how big that buffer should be if I don't know
the number of bytes waiting to be received. So there is a possibility that I
might not read out all the bytes and that some bytes will remain. So it's a
valid question to ask if I will get another OnReceive() call, as earlier I
was under the (wrong) impression that I had to read out all the bytes until
I got a 'would block' condition.
****
You don't care how many bytes there are. You read whatever you feel like. If there are
more bytes to be read, you will receive another OnReceive notification. WHat you do with
those bytes you read is up to you. TCP is a *stream* protocol, whcih means you will NEVER
know how many bytes there are to read; you will only receive notifications that there are
some bytes to read. End of concept.

If you need to assemble those bytes into some meaningful arrangement, that is up to you.
For example, I typically will preface a "message" in a stream with a 2-byte or 4-byte
length. I read the length bytes, which may take several read operations, allocate a
buffer as big as they tell me, and then on subsequent OnReceive notifications, read
however many bytes remain into the buffer at the appropriate position. WHen my "message"
is fully-assembled, I post it off to whatever is going to handle it. This is completely
transparent. I don't know, and don't care, how many bytes are waiting. It is completely
uninteresting to me. I just assemble messages and transmit them. Other approaches
include treating the data as "lines" and posting when a LF is seen (and retaining the rest
of the bytes for the next OnReceive to append to) or posting whatever came in.

If you think you have to care about how many bytes are actually there, you should rethink
your protocol, because it is incompatible with how TCP/IP works.
****

Perhaps the guy that said 'packets' should have said 'bytes of data'
instead, but I understood what he meant. Data has to be sent before it can
be received and I'm sure he meant that data from two separate sends are
received at two separate times and this might have some affect if Receive()
has not been called.
****
Data that is sent from two separate sends may or may not be received at the same time.
Short packets at the sender are coalesced into longer packets. There is ZERO correlation
between the number of sends and number of receives required. I can do a send() that sends
1MB, and that's a single send, and it could take a thousand recv()s to read it. Or I can
do 1000 1-byte send()s and get them all with a single recv(). You have no control over
any of this, and you should not care in the slightest.

TCP works by essentially using a distributed semaphore mechanism to implement flow
control. THe receiver says "I have this much buffer space" and the sender is free to send
however many packets will ultimately consume that buffer space. When the sender sees that
it has consumed the receiver's buffer space, it stops, and waits for the receiver to send
back a reply that indicates more buffer space is available. So all that matters is how
many bytes your current recv() receives. It correlates in no way with how many are sent.
All that TCP/IP guarantees is that *eventually* you *will* receive all the bytes that were
sent, in order, with no duplicates, or you will *both* know that the connection has
failed. But no guarantees beyond that are made. Such as time, correlation between
send()s and recvs(), where splits occur (if you have a 2-byte length, your recv() might
receive only the first of the two bytes, so be prepared to deal with that!), how long it
takes the data to get there, etc.
*****

I did further testing and changed to only one Receive() per OnReceive(),
instead of a loop. The problem has not occurred since. I switched back and
it occurred quickly. So it seems likely that this was the cause. What
confuses me is that my old way worked on several projects and even on this
project worked many many many times before failing. So reading all the data
until 'would block' works most of the time but not all the time. Since the
data sends, and therefore the data receives are asynchronous in this
project, I must assume the failure happens when something like an unlucky
timing coincidence between Send(), OnReceive(), Receive() and data being
received occurs. But since this is hard to prove, I would still like
confirmation from someone that knows for sure. I'd like to be 100% sure
rather than 95%. Thanks.
*****
I suspect it is a timing error as you observe. I suspect that this timing error is why
you should not try to do more than one recv() per notification.
*****

And Steve, I'm not sure where we both got the idea that all the data needs
to be read out in any OnReceive() call, but this latest test seems to say
that that is not correct.
****
Not only is this not correct, but it is *expected* that you will not read out all the data
from a buffer!
joe
****

Bill


"Joseph M. Newcomer" <newcomer@xxxxxxxxxxxx> wrote in message
news:iq8cn45e7nl3r8q3qq72448233vteqeumo@xxxxxxxxxx
See below...
On Tue, 20 Jan 2009 10:21:06 +0800, "Bill Brehm" <<don't want spam>>
wrote:

Scott,

I think there is no way to find out before the Receive() call how much
data
is waiting to be received, right? So I have to select an arbitrary buffer
size to read the data into. What happens if the amount of data I read out
is
not all the data that's in their waiting to be read? Will I get a second
OnReceive() call for the data that came in already that I failed to read
out?
*****
Yes. You get an OnReceive notification if there is input data to be read.
Why you have
concluded this has something to do with "packets" escapes me. It doesn't,
and never did.
Packets are some uninteresting low-level implementation detail that is
invisible to you.
****

My problem (in another branch of this thread) is that I seem to lose the
OnReceive() call rather than getting extra. I don't know if I am getting
extra calls (I will check this when i'm back at work soon) but if I am,
they
would just break out with the WSAEWOULDBLOCK condition and cause no harm
in
my case.

I will test with a single call to Receive() inside of OnReceive() and see
if
the problem goes away. But I would like to understand why the error is
occurring.
*****
I'm not sure why the error is occurring, but the explanations you are
suggesting are not
correct. There might be other reasons, but "packets arriving" have
nothing to do with the
problem.
joe
******

Thanks,

Bill


"Scott McPhillips [MVP]" <org-dot-mvps-at-scottmcp> wrote in message
news:%23838bDpeJHA.3708@xxxxxxxxxxxxxxxxxxxxxxx

"Stephen Myers" <StephenMyers@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in
message
news:5E3F541A-A5B5-4B47-8651-BA120F86CD9E@xxxxxxxxxxxxxxxx
Is there any posibility that the data is being received but not
triggering
OnReceive()?

This can happen when two (or more) packets are received without a
Receive()
call. If you are doing the Receive() in as part of the OnReceive()
handling,
make sure there is no data left prior to exiting OnReceive().

Steve

On the contrary. Every call to Receive enables the message that will
produce the next call to OnReceive. It is therefore recommended that
Receive should be called only once per OnReceive notification. It is
not
true that one must continue to call Receive until it would block.
(Reference: the MSDN page for WSAAsyncSelect - "an application need not
read all available data in response to an FD_READ message - a single
recv
in response to each FD_READ message is appropriate.")

And, if Receive is called more than once in OnReceive there is a
possibility that another OnReceive notification message (FD_READ) is
left
in the queue, and when it is processed later there will be no data.

Suggestion: Call Receive one time only per OnReceive, and make sure the
processing of received data does not permit the message pump to execute
until after you have returned from OnReceive. Both of these steps are
necessary to avoid the possibility of OnReceive and Receive getting out
of
sync.

--
Scott McPhillips [VC++ MVP]

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: Deaf CAsyncSocket on Windows Service.
    ... To read them out I need a buffer to ... the number of bytes waiting to be received. ... Perhaps the guy that said 'packets' should have said 'bytes of data' ... You get an OnReceive notification if there is input data to be read. ...
    (microsoft.public.vc.mfc)
  • Re: Losing UDP packets with MFC Sockets
    ... Receive you will pick up both packets in one buffer. ... >> Your receiving process does a Receive with a buffer of 100 bytes the ...
    (microsoft.public.vc.mfc)
  • Re: Deaf CAsyncSocket on Windows Service.
    ... Note that since you, at the application, have no idea how many packets are received, there ... the network stack to deal with the issues of reassembling a packet into a stream. ... how the receiver reassembles the input packets into a stream, ... Receive should be called only once per OnReceive notification. ...
    (microsoft.public.vc.mfc)
  • SIGIO + TCP communication.
    ... I used SIGIO signal when ever soket gets some information and i am ... receiver buffer on while loop but it able to read only few packets. ...
    (comp.os.linux.networking)
  • Re: In-tree version of new FireWire drivers available
    ... Just to recap, the dual buffer receive mode, as described in section ... quadlet aligned amount of header data can be appended into one buffer ... *either* the header buffer or the payload buffer fills up. ... enough to hold headers for all the packets it takes to fill up the ...
    (Linux-Kernel)