Re: output.c error in multithreaded program
From: Joe (not_at_home.com)
Date: 12/30/04
- Next message: Minus: "Re: Best Way to Add Custom Control to Dialog"
- Previous message: Joe: "Re: output.c error in multithreaded program"
- In reply to: Joseph M. Newcomer: "Re: output.c error in multithreaded program"
- Messages sorted by: [ date ] [ thread ]
Date: Thu, 30 Dec 2004 02:59:27 -0000
"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
news:nku5t05qu2vilpvi3jueqbdhsaic6h0art@4ax.com...
> On Mon, 27 Dec 2004 12:28:13 -0000, "Joe" <not@home.com> wrote:
>
> >
> >"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
> >news:81dvs0hgpp195fhu2usp9j6pj37i772srt@4ax.com...
> >> CWinThread::Create is essentially AfxBeginThread. Rarely is it
necessary
> >to create an
> >> instance of CWinThread and start it separately.
> >>
> >> pThread = (CThreadType1 *)AfxBeginThread(CThreadType1::ThreadFunc,
> >ThreadParams);
> >
> >> >Firstly.
> >> >#####
> >> >
> >> >I want to set up priorites for the
> >> >threads to access the internet - but I could find any examples so I
set
> >up
> >> >this structure using semaphores. I include the function desc which
should
> >> >explain my rationale
> >> >
> >>
>
>>dwRet=GetHttpFileHarness(LOW_PRIORITY,pThrPars->m_strServerName,pThrPars->
m
> >_
> >> >strObject, pThrPars,FALSE);
> >> ****
> >> I don't know what you mean by "priorities". Do you mean that requests
will
> >not be in FIFO
> >> order? That's easy: just modify your queue insertion routine to put the
> >request in the
> >> queue in priority order.
> >
> >Well I was trying to be clever. Dont forget there are several processes
with
> >several groups.
> >So I dont have a single queue.
> >The application sees type A header data as more important than C. I
didn't
> >want a process thread drilling a header page having to wait for 20 type C
> >threads to finish downloading their pages in another group or even
another
> >process entirely.
> >The other motivation was that I dont want to hit the server hard, I
restrict
> >the global semaphores to 6 in total - so I can only be making 6 requests
at
> >a time. A new type 'A' request may get stuck behind 100 type C requests
if I
> >can't have some sort of prioritisation.
> *****
> Then there's something wrong with your structure. Type A requests would go
in a type A
> queue, type C requests in a type C queue. Alternatively, a type A request
migrates past
> all type C requests in a queue until it hits a type A request. Thus you
implement
> priorities easily. But using thread priorities to try to implicitly
implement a notion of
> application importance is almost always a bad idea, because you are
working under the
> illusion that these priorities actually mean something. Think of them as
"suggestions"
> which the system may or may not honor. Because the Balance Set Manager
will boost a
> starved thread to priority 15 at random times, you don't have any way to
enforce the
> notion that all A threads will always preempt all C threads. Messing with
priorities is
> risky business. Every once in a while you need to, but it should not be
considered a way
> of doing business. It offers no real guarantees.
I don't have conventional thread priorites, just the "brute force" approach
above where I leave a couple of global semaphore openings for priority 1
request that priority 2 requests dont have access to.
> The problem is that there are several problems you are trying to solve
with fairly complex
> mechanisms. Instead, you need to reparition your program into simpler
tasks, and address
> the tasks. The mechanisms that fall out will be fairly straightforward.
>
> I've built some really complex multithreaded programs in my career, and
I've learned that
> simpler structures are better. Overall, I just try to keep everything as
simple as
> possible, working from the goal back to the mechanisms. Otherwise, the
complexity of the
> mechanisms blinds you about the goal.
>
> For example, one thing you've said is that there can be several processes.
A question is
> "why?" Why not a single process? But if the processes are independent,
then the number of
> them doesn't really matter; if you solve the problem once; it is then
solved for all
> processes.
> *****
Right, I did consider having one process, with the parent processes having a
queue(s) of URL requests to be executed at a time in the future. After a
request was executed the next one would be set up on a Timer.
But the problems I envisaged with this was firstly I thought I might hit
some system limits.
At an extreme I could have, say, 100 events. For each event I want to
download the state of the webpage at the exact same relative time to an
event "event". To use a ubiquitous example, say I wanted the stock price
on 100 stocks on exchanges around the globe at 10 minute intervals and
always *exactly* 1 minute before bourse closure. I *thought* I had more
chance of achieving this accuracy if I had several processes, one for each
bourse. They could all issue requests at the same time, only my global
semaphore restricting concurrent web requests (and of course the response of
the servers) would impinge on this accuracy. With just one process I was
unsure how it would cope with so much "going on" at the same time.
Note that the stock prices would be my class A requests. Depending on the
time to bourse closure and stock price we would want occassional snapshots
of a variable number of the most recent trades struck, say 5 to 40 - this
would be my class C. Once we have all the trades downloaded we can do some
stats or whatever on *all* the data. If the download succeeds I put each A
thread assigned to a stock to sleep until the next scheduled time (the C's
terminate) , if A fails (the site is down or whatever) I sleep for a shorter
period.
I give this scenario as an example becuase it's possible/likely that under
certain "time" conditions we could have a *lot* of pending requests.
So I could have gone either way, but my "procedural driven" rather than
"message driven" mindset coupled with the worries about "one process"
system limitations (say on the number of threads or maximum number of TIMER
requests) meant I took a guess and plumped for my "procedural" approach
along with multiple processes.
Throughout the whole development process I was consistently
wondering/worrying whether the one process approach, driven by the parent
thread/process and Timers would have been better- but I had no point of
reference on which to base any decision.
I don't want to start from scratch again with an "event/message driven"
unless I really really must. Also the HTTP
asynchronous method is much different to the synchronous one -
including having to use a callback function. I would basically have to go
back to the drawing board.
I dont know how to have one global queue for all the processes to allow me
to replace my two stage global semaphores prioritising and restricting
concurrent internet requests. Is it (efficiently) possible? (I suppose I
could go through a pipe to a single HTTP queueing/sending/receiving process,
but that seems to add another level of complexity).
For example, process 1 might have no type A requests but 10 type C. Process
2 might have 10 type A but no type C about to be sent. How can I ensure,
that the type A's requests are sent before the type C's and only say a
maximum of 6 requests are active at any time (as I don't wish to have any
chance of "overloading" the server)?
I will probably do all the changes over time and develop a one process
approach if, given my application specifics detailed above, you still thing
it's the way to go - but I just need something *working* ASAP.
Cheers
- Next message: Minus: "Re: Best Way to Add Custom Control to Dialog"
- Previous message: Joe: "Re: output.c error in multithreaded program"
- In reply to: Joseph M. Newcomer: "Re: output.c error in multithreaded program"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
