Re: output.c error in multithreaded program

From: Joe (not_at_home.com)
Date: 12/30/04 

Date: Thu, 30 Dec 2004 02:07:56 -0000

"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
news:smp5t01s3up68hpsiiqqj6stjol3s1s2l0@4ax.com...
> See below...
>
> On Mon, 27 Dec 2004 12:27:19 -0000, "Joe" <not@home.com> wrote:
>
> >
> >
> >Yes this area is another major source of confusion for me. I have tried
to
> >treat the internet specific HTTP stuff as a "black box" by integrating
the
> >demo code from the codeproject article "as-is". I think it is when the
site
> >is slow to respond that I get into some sort of "hang" scenario and the
> >GlobalSemaphore ends up blocking any other attempts at site access by
other
> >threads/processes.
> >Isn't the demo code asynchronous or do you mean something else?
> *****
> I'm not familiar with the example, so I can't answer that question.
> *****

What I have discovered (I have another question unaswered on this NG about
the subject) is it appears the timeout functionality does not work as (I)
expected - if at all. It only times out whilst waiting on a message
request - not if the server is down (I don't know how to test this
scenario) so a synchronous HTTP thread *can* hang indefinintely. But
actually, I don't *think* this is the problem, The program crashes when I
have several processes working under heavier than normal load and I reckon
the failure is down to the more typical "bad practise" scenarios you have
painted to me below and elsewhere.

> >
> >As I only get occassional errors and then only when I am running multiple
> >threads/processes and the target site is slow (I think this is the
scenario
> >for the "hang" type error I get anyway). I do not see the error
immediately.
> >I find an access violation windows message and start the JIT debugger to
> >find out what went wrong. The trouble is quite often the JIT quits with
an
> >error before getting in to the source code!!!!
> >So it become very hard for me to know how to track this error down and I
> >have ended up just hoping it would "go away" - buit it hasn't :)
> *****
> I've not seen the debugger fail to come up. The problem with this sort of
timing-dependent
> error is that it is indicative of some deeper bug, whether one of storage
allocation,
> thread management, or concurrency, as the most typical causes. Your
logging of events is a
> good start on looking for the conditions.
>
> Things I would look for:
> Concurrent access to shared structures from multiple threads without
> proper synchronization.
>
> Using variables in a CWinThread, particularly pointer variables,
> after the CWinThread has been deleted.
>
> Suggested technique: Set the m_bAutoDelete flag to
> FALSE. If the bug "goes away", you have this sort of
> error. Note that you will be leaking storage like crazy
> during this diagnostic procedure, but if the bug goes
> away, you've isolated a cause. Then you can fix it
> and let the m_bAutoDelete remain TRUE again.
>
> Standard dangling-pointer problems (using an object after it has
> been deleted), which just happen to show up on slow-response
> connections (a generalization of the above problem)
>
> Uninitialized local variables leading to bad-pointer situations.
>
> Uninitialized class member variables leading to bad-pointer situations.
>
> Instead of using JIT debugging, just run the app under the debugger. This
should be more
> reliable than bringing it in after the fact.

What I find extra hard in this project is the fact that aswell as the thread
issues I have HTTP ones on top, I can't work out how to reproduce the
situation where a crash occured as I don't know what the internet
interaction were at the time. What I could do however, which would be
informative, is to use dummy downlaods and see if this reveals any
"structural" problems in the thread processing.

> In addition, you can consider putting a C++ exception handler in your
thread. While it
> isn't among the best of techniques, because you have to use "..." to
indicate that you are
> catching all exceptions, it would help you isolate the thread that is
causing the problem.
> I've not done this in C++, only using C exceptions, so I'm not sure of all
the details.
> *****

OK I'll take a look.

<snip>

> >
> >Well, what I have is A in a loop, if the pages returned are "bad" - say
the
> >site is down - it sleeps for a while and then requests the header page,
> >which in turn generates another request for the detail pages (C via B).
> *****
> Another reason you should not be using syncrhonous models. The notion that
the thread is
> in a loop and waiting is also suspect. In a fully asynchronous model, you
would just queue
> up an event for a future time and process it at that point, without
needing to sleep at
> all. Note that when a thread is sleeping, no SendMessage to it will work;
the sending
> thread will block until the Sleep() expires, which then makes the sending
thread
> non-responsive to messages. The problem then cascades.
> *****

Yes, well, as I indicated elsewhere I couldn't work out *how* to guarantee
that a thread would receive a message - so I don't send any messages to
threads (I don't think I do anyway :) ).

> >I request A every n seconds/minutes in any case, depending upon
> >circumstances I may generate requests for C's. I looped A because I
thought
> >it a good enough approach to handle repeated requests.
> >
> >The volume I'm talking about is on an average day, is say 5 processes
(could
> >be treble this on a "busy" day) each with 15 type A's (could be thirty).
> >Several 'A's in a process can be awake at any one time. Each A can
request
> >data at a maximum of say once per minute.
> >
> >If it didn't sleep but just terminated I would have to set up some sort
of
> >structure which "knows" the status of each A(failed last time etc) and
when
> >to kick of type A's. I saw this as being just as complex. What problems
does
> >A waiting then sleeping cause?
> *****
> Actually, if you think of "service threads" as servicing requests, you
don't need to keep
> creating them. You now have a "thread pool" model. This is actually
simpler than models
> that sleep, because the thread is always working unless there really is
nothing to do.
> I've used this technique several times to handle this sort of problem; for
example, I have
> a case in which I have to resolve IP addresses and re-establish a
connection. The remote
> site may be down and non-responsive. So I basically queue up requests to a
connection
> handler. What the connection handler does is handle an event that comes in
to say "Wake up
> and handle an event". It dequeues the first event on the queue, and
attempts to establish
> a connection (asynchronously). That's all it does. At some future point,
the connection
> either succeeds, in which case it passes it off to the traffic-handling
thread, or it
> fails, in which case it schedules a future attempt. It starts with
5-minute intervals,
> then goes to 10, 15, and 30 minute intervals. All threads were very
simple, and the
> scheduler thread (done as a UI thread) was handled with WM_TIMER messages,
because that
> was the easiest way. I just did a SetTimer that reset the time so that the
next WM_TIMER
> message came in at the right time to trigger the next event.
>
> THis is one case where I can't show the code, because someone else
actually owns the
> rights to it. Otherwise, I'd publish it.
> *****

Yes I did *think* about attempting this type of structure. Please see my
reply to your (19.01) post.

Relevant Pages

  • It useded, you refered, yet Youssef never simultaneously received on board the calendar.
    ... The massive manuscript rarely occupys Abbas, ... in connection with mad courages. ... I was needing aggressions to severe Abdel, ... Otherwise the sleep in Yosri's imprisonment might resist some ...
    (sci.crypt)
  • Re: Connecting Out of Process Servers via COM+
    ... connection, and in fact we can only use one connection because of the way ... years ago when I needed a number of clients to share a single serial ... it would create a Dispatcher-Object first and registers ... > This means, altough COM+ would allow parallel requests, the calls would be ...
    (microsoft.public.vb.com)
  • Slow DNS requests?
    ... I think that the delay is in resolving the DNS requests. ... Guest machines connected via CAT5 to the BEFSR41 run fine. ... As soon as the host name is resolved, performance seems to improve dramatically for that connection. ... It offers one set of entries for "Name Servers" and a different set for "Domain Search". ...
    (comp.os.linux.networking)
  • Re: http pipelining
    ... Oops, sorry, you meant sending requests in parallel, right? ... down a single TCP connection, without waiting for the first response. ... Certainly urllib and urllib2 don't support pipelining. ...
    (comp.lang.python)
  • Re: ActiveSync 4.1 and beta 4.2 doesnt work..
    ... the device will noty go to sleep actually. ... If you go to Activesync App, there is a connection Settings menu option ... When you cradle the ...
    (microsoft.public.pocketpc.activesync)