Re: Async socket & active connections

Tech-Archive recommends: Fix windows errors by optimizing your registry

On 25 Mar, 21:48, "atlaste" <atla...@xxxxxxxxx> wrote:
Hi,

In an attempt to create a full-blown webcrawler I've found myself
writing a wrapper around the Socket class in an attempt to make it
completely async, supporting timeouts and some scheduling mechanisms.
I use a non-blocking approach for this, using the call to 'poll' to
support the async mechanism - rather than the 'begin' and 'end'
functions. I already found that connecting doesn't set the
"isconnected" variable correctly (SocketException is thrown: non-
blocking has this effect...) - but doesn't appear to be a problem
because poll, read and write work fine.

For measuring the performance of the crawler, I started "perfmon.msc"
and added the "active connections" item from object "TCP". After a
while I found the number of this performance counter to reach over
300K connections (!), enough to start worrying...

My crawler is designed to support around 200 connections simultaneous.
"netstat -an" doesn't support this finding, but does show hundreds of
connections that are in either "CLOSE_WAIT", "FIN_WAIT_2" or another
closing state.

After a host has completed, I try to disconnect the TCP/IP connection.
I've attempted combinations of "shutdown(both)", (async) "disconnect"
and "close(0)" - where no combination appears to have the desired
effect. When the application is shut down, all connections (including
the CLOSE_WAIT connections) are removed. The FIN_WAIT_2 connections
linger forever...

Perhaps someone knows a solution to this problem?

Greetings,

Stefan de Bruijn.

Just a guess, can you try it with keepalive set to 0?

Something like where s is a Socket:

s.SetSocketOption(System.Net.Sockets.SocketOptionLevel.IP,
System.Net.Sockets.SocketOptionName.KeepAlive, 0);

.

Relevant Pages

  • Re: Industry Standard Security and guest wifi access best practice
    ... with IPSEC VPN clients has not been positive. ... Then they probably won't support other forms of security. ... to switch all connections into SSL mode. ... Use WPA to encrypt wireless traffic, ...
    (alt.internet.wireless)
  • Re: Async socket & active connections
    ... completely async, supporting timeouts and some scheduling mechanisms. ... I use a non-blocking approach for this, using the call to 'poll' to ... support the async mechanism - rather than the 'begin' and 'end' ... 300K connections, enough to start worrying... ...
    (microsoft.public.dotnet.languages.csharp)
  • Async socket & active connections
    ... completely async, supporting timeouts and some scheduling mechanisms. ... My crawler is designed to support around 200 connections simultaneous. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Yes - Off-Line Temporarily
    ... Apache will give you more connections, but will not support the server ... Microsoft MVP - FrontPage ... security concerns at http://www.remotedirector.com. ...
    (microsoft.public.frontpage.client)
  • Success Re: Help and Support Center Problem
    ... My problem in brief: In the Help and Support Center, the Help viewer had these symptoms: 1) the font changed to Times New Roman, 2) links no longer worked, 3) the H & S graphics did not appear and 4) certain lines and phrases were replaced by XOX. ... I'm still left with the minor annoyance that H & S doesn't think I'm connected to the internet. ... That's no big deal, really, because I can still connect to the MSKB and Windows Update through H & S. In fact there are two MSKB articles that describe this problem, but both refer to DSL connections and I have cable. ... > I've tried System File Checker, replacing hh.dat and reinstalling H &> S, ...
    (microsoft.public.windowsxp.help_and_support)