Re: IOCP touching my OVERLAPPED after my Write has completed



"Alexander Grigoriev" wrote:

Are you actually calling GetQueuedCompletionStatus in your completion
routine that you pass to WSASendTo? That would be an error.

No I'm not.
DON'T mix a completion routine and IOCP thread, those are different
concepts.

I'm aware of the difference.

thx for your time
Alberto
"Alberto Demichelis" <AlbertoDemichelis@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in
message news:FFCA14D3-4C2A-448F-B211-3187D1018C79@xxxxxxxxxxxxxxxx


"m" wrote:

The first issue I see is that the return code from GQCS can be zero and
still return an IORequest that will need to be freed - this usually
happens
if the network fails. Your code will leak memory in this case.

I handle all the error case as aborted OPs etc... I tried to keep the
pseudo
code small.



The short answer is that when the call to GQCS returns the buffers
(OVERLAPPED & data) can be freed immediately.


Ok, this is what I thought. But looking at my traces + disassembly of
WSASendTo() it seems a bit different. It looks like WSASendTo is
implemented
like this

Here I'm guessing:

int WSASendTo(...OVERLAPPED *lpO,..) {

IssueWrite(lpO);

if(lpO->InternalHigh) { /// here I get the assert
..... stuff
}
stufff
}

So seems that if my write completes before IssueWrite() returns then the
following line will generate the read viaolation because my buffer has
been
uncommitted by the completion routine.

My trace looks like this (pseudo trace)

[Thread id]
[0x001] WSASendTo Alloc(0x1185d000) <-- app thread
[0x002] Completed Free(0x1185d000) <-- worker thread
[0x001] assert violation at 0x1185d004 <<-- this crash in the send to
because on my uncommited page. is the OVERLAPPED::InternalHigh

Is this wrong or I'm just immagining things?

ciao
Alberto


"Alberto Demichelis" <AlbertoDemichelis@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote
in
message news:4A9CBAEE-6F48-40F8-AF2A-023277E88CD3@xxxxxxxxxxxxxxxx
"m" wrote:

No you have not misinterpreted the MSDN documentation. After the call
to
GQCS returns the IO operation is complete and the memory associated
can
be
freed or reused.


Ok, I think here we are all talking about the same thing. Just to be
sure,
here is what I do.

struct IORequest : public OVERLAPPED {
...stuff...
...data...
..
}

Main Thread ---------------------------

IORequest *req = AllocateRequest(); //commits a virtual page and
initializes
stuff
WSASendTo(socket, ..., req,....);

WorkerThread(aka completion routine)----------------------------

if(GQCS(req)) {
FreeRequest(req); //in debug I decommit the memory of this request
}
else
{
....error handling...
}

This seems to be unsafe. Is how am I supposed to do it instead? How do
I
know when I can delete my Overlapped+Buffer? That's all I ask.
Just point me to a book or a article I do not want to bother you too
much(not Jeffrey Richter's please because I have those books and is I
just
not clear enough for me on this topic).

ciao
Alberto


Anton is telling you that the memory must not be freed (or accessed)
between
the call to WSASend etc. and the return of GQCS.


"Alberto Demichelis" <AlbertoDemichelis@xxxxxxxxxxxxxxxxxxxxxxxxx>
wrote
in
message news:5444A636-B03E-4283-BD9C-7A7BEC8938BC@xxxxxxxxxxxxxxxx
First, thanks for all the time you are spending on my issue.

I must have misinterpreted this sentence from MSDN:
"The lpOverlapped parameter must be valid for the duration of the
overlapped
operation."
I supposed the completion of the request is the end of the
overlapped
operation.
But if this is the expected behaviour then they should call it
'almost
completion routine'.

The case 2 you are proposing, is exactly what I'm doing and is the
source
of
my 'bug'. The fact that I protect the memory is just to debug the
system.
I cannot mark the memory as free and put it back in my pool when the
IO
completes. If another worker fretches it back from the pool before
winsock
is
done with it, bad thing could happen(guess).

To mee seems that I have to emply some form of refcounting to be
safe.

ciao
Alberto

"anton bassov" wrote:


What are these samples?????

Try to look at the situation from the caller's (i.e.Winsock) point
of
view. Once you have passed a perfectly valid pointer to some
function,
you may want to check the pointee
after the function returns. Therefore, it would not be too
unreasonable
for you to expect
this pointer to remain valid upon the function's return, don't you
think?????? If you are still not convinced, I have to remind you
that
in C language we pass arguments by value,
so that we expect them to be immutable. The only thing you can
change
is pointee, but pointer in itself should stay the same. In other
words,
what you are doing is very, very bad programming technique in
itself,
and you should avoid doing things like that.

In your particular situation I see 2 possible solutions:

1. Don't use the completion routine at all - just pass unsignalled
event in OVERLAPPED structure, and make some other thread wait on
it.
When IO operation gets completed, this event will get signalled,
and,
at this point, you will be able to safely free memory


2. Don't free memory, but, instead, just mark it as available - you
will be able to reuse it upon the next send or receive operation.
As a
result, you will avoid unnecessary calls to VirtualAlloc() and
VirtualFree(), and, hence, improve your performance (this is
particularly true if your program sends and receives data
frequently)

Anton Bassov



Alberto Demichelis wrote:
yes I'm decomminting in the completion routine. I've seen that
all
examples
do some form of release in there.

Alberto

"anton bassov" wrote:

Hi Alberto

When the request completes I uncommit the page

Are you doing it in IO completion routine?

Anton Bassov


Alberto Demichelis wrote:
Hi, I've recently posted about a crash I was experiencing in
my
server on
heavy load.
Recently I've rewritten part of my IOCP routines and added
some
extra
consistency checks. While doing this I've
realized that the my OVERLAPPED struct gets read by winsock
after
the IO
request has completed. That in my current implementation
means
that
is
reading freed memory.

This is my scenario. I have a IO request pool implemented
using
VirtualAlloc.
I reserve few megs of virtual memory, every time I allocate a
request I
commit a page and put all my IO structs in it.
When the request completes I uncommit the page.

The result is an access violation in WSAWriteTo() [Access
violation
reading
location 0x1185d004]. This happens when the IO request
completes
before
WSAWriteTo() returns.
Probably the same happens in my ReadFrom but apparently it
never
gets
triggered beacuse I deallocate my read requests in my main
app
thread after
processing.

Now I'm a bit lost, all the IOCP samples I found on the net
use
the
same
pattern I use. When can I safely
release my OVERLAPPED and related buffers? I thought was 'on
completion' by
apparently I'm wrong.

thanks for your time
Alberto













.

Relevant Pages