Re: Threading a server

Hi Peter,

Thank you very much for your response and your comments. I do take them on
board. This is getting quite long and not sure if you will read it all, but
I have comments / questions towards the end about some of the technical
issues that you raise. I have hightlighted them starting a line with
*****************

The code I have given is almost complete. All I have missed is some
supporting functions (like parsing the incoming lines from the socket,
logging, and handling the resulting message (in the SMTP handler)) but
everything else is there that is required. There is a POP3 handler inside
the main part of the service, but this code is not referencing that at all.
There may be one or two variables that I have missed out, which mostly get
information from the app.config. The variables that are being sent to
handler are just variables being put into properties. These are for the
ancilliary functions after DATA has been handled (and those functions work
fine, as they are the same as the ones for the POP3 handler and the POP3 is
fine.) LogToFile and LogVerbose are just boolean values coming from
app.config as to wether the app should be logging.

With respect to my clients. I told them from the outset. They know I am a
web dev and have used me for that purpose in the past and like my work. I
have also done simple windows services for them. For this project, it
required a mailserver, purely to receive incoming mail. I had a working POP3
mailserver but they wanted one that was more pro-active, which only left
SMTP. Now, I have written a mailserver in the past in Delphi (in my early
days of programming) and I used TWSocket, which is event driven rather than
thread driven.

However, I could not find anything like TWSocket for C# and the only SMTP
sample I could find was the thread driven one.

I was not aware that thread.abort would cause a problem. I was reading a
book on threads and threading (to try and get my head around threads) and
this watchdog was mentioned. It seemed like a good idea, but only recently,
I realised it might cause other problems (i.e. it may not close the socket
or do any of the cleanup). For this, I searched google for "c# abort a
thread from the inside" but nothing really jumped out at me.

When I ran the server I was getting 100% CPU. I did a search for "c# thread
taking 100% cpu" to see if it might lead me in the right direction.
Obviously not. The one or two messages I found suggested using thread.sleep.
I didn't like that. I felt it was a hack, but as I didn't know any better...

Basically, I did as much as I could to resolve this issue myself and to
learn about threading. It was over 3 weeks ago since I asked the initial
question and I have been working on this for quite a bit of that time,
including long evenings and weekends (at no cost to the client, and the
client is aware of my dedication to this) to make this work. However, the
fact is that in this particular case, there was too much for me to learn and
I was under time pressure. I didn't really want to do the SMTP side, but I
did and tried to use the experience as a learning exercise. (What didn't
help is that a week into this project, my laptop hard disk died and so did
my backup hard disk, both at the same time, so was without my computer for a
week whilst I tried to resolve the issue.)

I have tried many threading examples to make this work. This final one
appeared to work for me, and I understood what I was doing with it more than
the other ones. As it turns out and with your comments to me, I have grossly
misunderstood.

When I tested this version of the code against problems I had in the past
with it, this appeared to work. It worked with my multiple telnet sessions
and it worked when I sent multiple emails from outlook express direct to it.
However, it failed when I put it on the live server.

I do have to learn and obviously I am making mistakes that I don't know how
to get out of. My research into it is leading me nowhere, hence my
questions. I don't ask simple questions, as I feel if it is that simple,
there will already be an answer I have overlooked, and quite often, I am
right. Hence the reason I am here. Threading at the moment, whilst may be
simple for you is being a nightmare for me.


*****************

With respect to your further answers on the topic...

The 100% CPU. I couldn't understand that either, but it happened on both my
machine and the server that I uploaded it to. OK, It might have only been
97 - 98% CPU, but still heavy load. There was no load intially. If I
connected one telnet and closed it, it was fine. If I connected two, then
closed one, it would 100%. To me, this meant that something was looping and
I couldn't figure out what. (I stuck breakpoints in my loops but they
weren't hit.)


On my slightly earlier version of this code, I only had one thread set up.
This was being blocked by the Thread.Join and was stopping any further
connections. I commented out the clientThread.Join, clientThread.IsAlive and
clientThread.Abort and everything appeared to work. However, I felt that I
needed the watchdog and with a reasonable amount of time (3 minutes). The
only way I could think of to get it to work is to put that thread in another
thread, so that the Thread.Join was not blocking any other threads. Perhaps
this is what was causing my 100% CPU.


The SMTPServer.Run taking an object that is not used. This was a bit of junk
left over from another attempt to run it without blocking (AcceptTcpClient
blocks it). The thread stuff for that was ParameterizedThreadStart, which of
course, passes a value. I couldn't start the thread without it, so put it
in, even though I never used it.


The timer that you mention. Would that be System.Threading.TimerCallback,
and would that be on the outside (i.e. in the main service rather than
inside the SMTP class). When would I initiate the timer? Would it be just
after I start the thread in the while(true) loop inside ListenForClients, or
would it be inside the thread that has just been started? Also, how would I
tell the timercallback which thread (or TcpClient) I need to finish (or will
it just know?). I am guessing that to deal with the timeout behaviour, the
timer has to somehow call into the thread to stop things and run a cleanup.
From what I see of threads, when I start one, as I am not naming a thread
(that I can see, as they all seem to have the same name) I don't know how I
can connect to that particular thread.


Mostly when I use exception catching, I will log it to a file and get the
Message and Stack Trace (and sometimes the values in certain variables). In
general, I don't use exceptions unless I specifically need them. Now, in
this case, I was getting an exception and due to my lack of understanding of
threading, I put it in to attempt to work out what was happening whilst
still keeping the rest of the process going.


With respect to the SMTPServer class, my thoughts exactly, but I posted it
to give as much information as possible (though you still say I am short of
code) to help me to resolve the issue.


The next message you posted commented on the loop. Putting that loop in I
guess was a mistake (and could be the cause of most of the problems). I only
realise that the loop could be an issue as you have just pointed it out. As
I am now using other threads to maintain an unblocked system, I guess this
loop is no longer required (as I have the loop in ListenForClients), as all
it will do is to start the child thread, wait for the timeout of
thread.join, timeout the connection then restart it again (which would
explain why after timeout, I get the SMTP banner message again in my telnet
window). Could it be this that is causing sendmail (from linux) to be having
the problem of sending the next message?

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available
"Peter Duniho" <NpOeStPeAdM@xxxxxxxxxxxxxxxx> wrote in message
news:op.uswwxzy68jd0ej@xxxxxxxxxxxxxxxxxxxx
On Fri, 24 Apr 2009 06:07:41 -0700, David
<david.colliver.NEWS@xxxxxxxxxxxxxxxxxxxxxxx> wrote:

Hi all,

I still got problems with it...

I have sorted out the watchdog process by having a parent thread. May
not be
the most elegant. I then experienced 100% CPU so have put in
thread.sleep to
sort that.

However, when I test locally, it all appears to work, though when I put
it
live and sendmail is sending to it, it receives first time and if the
next
job comes within my timeout period of 3 minutes, it hangs.

Here is my source... Any help at all would be appreciated. This is in a
windows service.

I am getting myself into a really big hole with this and the client is
(to
put it politely) a little upset. :-(

All due respect, you've posted quite a lot of code, and yet not actually
complete, expecting others to become familiar with yet another code
example you've downloaded, and are asking for those others to solve your
implementation problems with that incomplete-but-inconcise code example.
That's asking a lot.

Hindsight being 20/20, I'd say it was a mistake for you to offer your
professional services with respect to a threaded SMTP server, without
having any actual knowledge about how to use threading in .NET. I'm sorry
your client is becoming impatient, but you may find that you get the most
respect from him at this point by confessing your sin and acknowledging
that either he simply needs to let you take the time you need to learn the
API (at your cost, not his), or he should hire someone else with more
experience in the area.

Now, all that said, if I have time later, I'll take a closer look at the
code you've posted. It would be better for you to post a
concise-but-complete code example that reliably demonstrates the problem.
But if that's the best you can offer, I suppose it'll have to do.

For the moment, I'll suggest a threading rule of thumb: if you are calling
Thread.Sleep() for any reason other than to wait for some specific,
known-in-advance time, you've made a mistake. By "specific,
known-in-advance time", I mean that there is something else going on that
you _know_ will take a certain amount of time, or you specifically need
the thread to wait for a specific amount of time.

For example, you've got some kind of simulation thread that is supposed to
update the simulation every 100 ms. Or you've just queried some physical
process known to take a fixed amount of time, and the thread needs to wait
until that fixed amount of time is done.

In the code you've posted, you've got these 1/4 second calls to Sleep()
sprinkled about like fairy dust, as if they will magically make some
perceived problem go away. They don't belong.

Also, calling Thread.Abort() is not an appropriate way to interrupt
processing. For threads that are actually doing some processing, you
should use an appropriate signal (sometimes this is as simple as a boolean
flag). For something like this, where a thread is likely blocked on i/o,
you should close the i/o object, causing an exception to be thrown from
the i/o operation being executed. In either case, this allows the thread
to terminate gracefully, cleaning any necessary data up and ensuring
consistent execution.

I'm guessing that your "it hangs" problem is a result of a thread being
tied up doing something it's not supposed to be doing, but until I have
more time to look more closely, I won't be able to say for sure.

In the meantime, hopefully the above gives you some food for thought.

Pete


.

Relevant Pages

  • Re: Form OnKeypress event not firing while test loop running
    ... Paul E. Schoen wrote: ... Once this happens the display shows the running time and current value, and this may be repeated until a certain number of pulses occur, or a preset timeout is reached. ... The data analysis is accomplished in a rather large loop which fires on a 200 mSec timer event, and it processes about 480 integer data values. ... ESC key for example, then while your in that loop, you can test for that value incase the user wants to terminate the current function.. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Need Help in Loop Foreach
    ... I'd like to insert a loop, in such a way that it reads all the "System ID" ... # Query all the systems status by inputting the SystemIDs on Expect Script ... intial expect until a timeout occured. ... # may want to break after sending exit in above section if this does ...
    (comp.lang.tcl)
  • Re: creating a new thread to run while loop
    ... > connection suddenly dropped, so i have to go for an alternative which ... thread) is not able to timeout / return error / etc and is basically being ... The only options seems to be to terminate the ... The example that you supplied w/ the loop in the main thread that causes the ...
    (microsoft.public.windowsce.embedded.vc)
  • TimeOut Expired. Using Temporary tables
    ... I am iterating through these records in a WHILE Loop and performing certain business rules on these records. ... This entire processing is being done in a stored procedure which is being executed using ADO.NET's ExecuteNonQuery method. ... However after around 18 hrs of processing time i get a TimeOut error. ... Post Made from http://www.DotNetJunkies.com/newsgroups Our newsgroup engine supports Post Alerts, Ratings, and Searching. ...
    (microsoft.public.dotnet.framework.adonet)
  • Re: bootimage.c (3)
    ... what I doubt is that since the 'while' loop ... reads in click_size quantum if we don't have a rounded amount, the addr (as ... position and not to the end of the data segment, ...
    (comp.os.minix)
Loading