Re: Parallel processing questions

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Paul fpvt2 (anonymous_at_discussions.microsoft.com)
Date: 02/28/05 

Date: Mon, 28 Feb 2005 14:28:38 -0800

Thank you very much for your help.
Please let me know if I understand it correctly.
1. If I need to send big data and it is going to take
a while, then it might be best to use ActiveX exe that
calls the SendData method.

If the data is not big, but I need to send the data very
often (like thousands of time in 1 seconds), it might be
best to do everything in the main EXE and call SendData
method from it, instead of from ActiveX exe.

2. In my case, the data is small (probably around 500
characters), but I need to send this to multiple clients
very often (maybe thousands of time in 1 seconds).
So, do you suggest to put process everything in the main
EXE and call SendData method from it?

3. You mentioned earlier that "when one of your app
clients requests "all records", the server socket queued
over a million records to be sent and started to send
them. When another client asked for its email, it worked
it in. "
Did you mean that while the server socket is sending
million of records to 1 client, it can, at the same time,
accept request from other client ?
I do not see that from the codes that you posted. Where
does it do the parallel processing in the codes ?

Thanks.

>-----Original Message-----
>Since TSX-32 rose and fall, so did virtual parallel
processing. I miss it
>dearly.
>
>If you load 100 instances of a standalone EXE program,
then the processor
>will take turns executing the stack from each of those
100 instances, in a
>first Data come, first Data serve kind of sequence.
>
>vs.
>
>If you have 100 Winsock instances inside of one EXE, and
you manage them
>right, then the single EXE will execute the stack from
each of those 100
>instances, in a first Data come, first Data serve kind
of sequence. So the
>question becomes, which is easier to write, uses less
memory, easier to
>comment, easier to expand or adapt in the future.
>
>There was a point in time where I farmed tasks out to
other EXE's, but
>learned to manage the code a little better and brought
those functions back
>into the main EXE, which actually simplified
everything. There are
>advantages to loading external EXE's to perform work,
but this doesn't sound
>like one of them. If you have an instance where
something is going to take
>awhile, then it might be best to farm it out to an
external EXE that way you
>don't have to worry about what it's going to cut into as
far as time slices
>go, but even that isn't entirely literal. Here's what I
do, (code snippets
>follows in order to express an idea).
>
>In this case, I manage a SMTP connection for each socket
instance, and at
>design-time I have no idea how many there will be at run-
time ... maybe 1
>(not likely), maybe 100 (more likely).
>
>These are the Equates that will be arrays (0 to # of
sockets), and since you
>are managing server and client modes, you can make it
all one module if you
>want like I did.
>--------------------------
>Private Type SMTPUser
> MODE As Byte
> REQUESTID As Long
> AdvHostName As String
> RealHostName As String
> IPNUM As String
> SMTPFrom As String
> SMTPMailText As String
> SMTPRcpt As String
>End Type
>
>Private Type SMTPRemoteServer
> MODE As Byte
> SMTPFrom As String
> Rcpt As Recordset
> SMTPMailText As String
> MessageNumber As Long
> TargetServer As String
> DataReady As Boolean
>End Type
>
>'Global
>Private MaxConnections As Integer
>'Constants to hold what Mode each socket is in so I know
what commands
>should be expected or ignored
>Private Const wskINITIAL As Integer = 0
>Private Const wskINITIAL1 As Integer = 1
>Private Const wskINITIAL2 As Integer = 2
>Private Const wskREADYDATA As Integer = 3
>Private Const wskRECORDING As Integer = 4
>Private Const wskRCPT As Integer = 5
>Private Const wskDATA As Integer = 6
>Private Const wskGOODBYEDATA As Integer = 7
>Private Const wskGOODBYE As Integer = 8
>
>'Connection Specific
>Private SMTPConnection() As SMTPUser 'Maintains
Connection specific
>data
>Private currentDatastream() As
String 'The datastream
>being built
>Private NeedsAFriend() As
Boolean 'Still has not entered
>a valid FROM or RCPT
>Private tempparm() As
String 'Holds a
>parameter
>Private FoundValidUser() As
Boolean 'Found a target RCPT
>value in local mailboxes
>Private currentStream As
String 'Holds inbound
>data between Function calls to assemble
>Private SPAMDETECTOR() As Boolean 'Has
triggered SECLEVEL1, goto
>TRAP mode
>Private NULLCount As
Byte 'Start Criminal
>signature profiling
>--------------
>Ok, this stuff is managed by all the functions and subs
and has no problems
>at all managing connection (or Socket) specific data for
each connection.
>Now if the design is tight, we don't care wht order
stuff is received in, or
>what order it goes out by "time-slicing", and we don't
care who does the
>"time-slicing" since somebody HAS to do it. All we have
to worry about is
>bogging the code down, which would eventually hose
everybody simultaneously.
>
>The rest is VB 101 kind of stuff, use indexes and farm
out time gobblers.
>When you get a phone call, take the opportunity manage
your sockets:
>If the socket has something to say, say it as fast as
possible.
>--------------
>Private Sub wskSMTP_ConnectionRequest(Index As Integer,
ByVal REQUESTID As
>Long)
>
> Dim loopvar As Integer
> Dim AllFull As Boolean
>
> AllFull = True
> 'Find the first "Offline" connection and tell it to
answer
> '#0 is the listener, we can't tell it to accept
> For loopvar = 1 To wskSMTP.UBound
> If wskSMTP(loopvar).State = sckClosed Then
> AllFull = False
> NeedsAFriend(loopvar) = False
> wskSMTP(loopvar).Accept (REQUESTID)
> SMTPConnection(loopvar).REQUESTID = REQUESTID
> SMTPConnection(loopvar).RealHostName =
>wskSMTP(loopvar).RemoteHost
> SMTPConnection(loopvar).IPNUM = wskSMTP
(loopvar).RemoteHostIP
> SMTPConnection(loopvar).MODE = wskINITIAL
> SPAMDETECTOR(loopvar) = False
> If GetINIi("SMTP", "Log", 0) Then
> AddSmtpLog ("Accepting connection from "
& _
> SMTPConnection
(loopvar).RealHostName & " >From "
>& _
> SMTPConnection
(loopvar).IPNUM)
> End If
> Call SendSMTP("220 Welcome to SMTPot(tm)
version " & App.Major &
>"." & _
> App.Minor & "." & App.Revision & "
SMTP Socket #" &
>loopvar & _
> " of " & wskSMTP.UBound -
wskSMTP.LBound, loopvar)
>
> Call ChangeSessionItem(FormSessionParm
(loopvar, 0) & "
>Connecting...", loopvar, 0)
> loopvar = wskSMTP.UBound
> End If
> Next loopvar
>
> If AllFull = True Then
> wskSMTP(0).Close
> wskSMTP(0).Accept (REQUESTID)
> If GetINIi("SMTP", "Log", 0) Then
> AddSmtpLog ("Refusing connection (Full)
from " & _
> wskSMTP(0).RemoteHost & " From "
& _
> wskSMTP(0).RemoteHostIP)
> End If
> Call SendSMTP("+ERR Connections full. SMTPot(tm)
version " &
>App.Major & "." & _
> App.Minor & "." & App.Revision, 0)
> wskSMTP(0).Close
> wskSMTP(0).Listen
> End If
>
>End Sub
>--------------
>You don't even care about remembering the socket #, if a
socket has
>something to do, it will pass itself to wherever it
needs to go. Now, if
>you've got something to say, just say it.
>--------------
>Private Sub SendSMTP(ByVal datastream As String, Index
As Integer)
>
> If wskSMTP(Index).State = sckConnected Then
> wskSMTP(Index).SendData (datastream & vbCrLf)
> End If
>
>
>End Sub
>--------------
>and Bam, it's gone. If something needs to move on to
something else, so be
>it. Doesn't hurt us any. Just remember to houseclean
(garbace collect) the
>sockets when somebody hangs up. If you need that, I can
send you a snippet
>for that too. Somebody is going to "work you into the
other needy
>processes". Manage it right, and you can kind of do it
yourself from one
>app, but at the very least get yourself to a point where
you don't need to
>worry about it anymore.
>
>I see what you mean by needing multiple processes to
send data out to
>multiple clients simultaneously, but there really is no
such thing as
>simultaneous, nor can one socket connect to multiple
targets at once. UDP
>would be your best bet then just because there is a LOT
less to manage, but
>it's still one computer at a time. With techniques like
that above, it will
>be as close as computers of our caliber can get, with no
headaches (once the
>bugs are worked out). This would actually be a bit
faster than multiple
>apps, simply because connecting and disconnecting takes
a pretty significant
>chunk of time when compared to the lifecycle of buffer
read/writes.
>
>Hope this help, usually when I really know the answer,
it's because I
>mis-understood the question.
>
>BT3
>
>
>"Paul fpvt2" <anonymous@discussions.microsoft.com> wrote
in message
>news:121e01c51da6$0674c620$a501280a@phx.gbl...
>> Thank you very much for your reply.
>>
>> >When the number of socket connections start to hover
>> >around 30 or so, best to farm out complex SQL string
to
>> >another application (read ActiveX EXE) altogether and
>> >let the processor figure it out.
>> Did you mean like the following:
>> Everytime I need to send data to clients, I will pass
the
>> server socket control to an ActiveX exe. In the ActiveX
>> exe I will also have a timer with interval = 1, so that
>> it will exit the ActiveX exe right away and let the
main
>> application be free to do other things.
>>
>> In the ActiveX exe I will call the Send method of the
>> server socket control to send data to each client
>> connected to the server socket control.
>>
>> If I do the above and pass the same server socket
control
>> to an ActiveX exe, will I be able to get parallel
>> processing for the server socket control ? For example,
>> if 1 of the ActiveX exe send data to 50 clients, can
the
>> 2nd ActiveX exe at the same send data to say 30 clients
>> (some of the clients might be the same with the 1st 50
>> clients) ?
>>
>>
>> >When it gets to 80 or so, best to just handle socket
I/O.
>> What did you mean by this ?
>>
>> >To give you an idea, when one of my app clients
>> >requests "all records", the server socket queued over
a
>> >million records to be sent and started to send
>> >them. When another client asked for its email, it
>> >worked it in. When 3 people sent email to it, it
worked
>> >it in. Server is a 2.4Ghz, but it worked equally well
>> >on my old P1 back in the day, just slower.
>> On the above scenario with > 1 million records to be
>> sent, did you use ActiveX exe ? Would you please kindly
>> post your sample codes for it ?
>>
>> Thanks a lot.
>>
>>
>
>
>.
>

Relevant Pages

  • Re: Issue with multiple threads and System.Net.Sockets.Socket
    ... first off your app has a 200 loop count, I am pritty sure you can only throw ... private thd as system.threading.thread ... private eip as string ... Are you closing the socket out? ...
    (microsoft.public.dotnet.languages.vb)
  • Proxy Server
    ... Private clientSocket As Socket ... Shared Sub MainAs String) ... Dim tcplistener As New TcpListener, ...
    (microsoft.public.de.german.entwickler.dotnet.vb)
  • Re: Threads Vs. Handles
    ... Private Sub Listen_Callback ... Private _Socket As Net.Sockets.Socket ... Public Event Disconnected(ByVal ID As String) Implements ...
    (microsoft.public.dotnet.framework)
  • Re: Threads Vs. Handles
    ... Private Sub Listen_Callback ... Private _Socket As Net.Sockets.Socket ... Public Event Disconnected(ByVal ID As String) Implements ...
    (microsoft.public.dotnet.framework)
  • Re: Applet Hangs when submitting data to servlet
    ... to put a time limit on my socket connections and socket reads. ... public void setTimeoutthrows UnknownHostException, ... on our web server. ... private JButton theSubmitButton_, theClearButton_; ...
    (comp.lang.java.programmer)