Re: Large virtual memory, small working set

Have you considered the application design, I mean, is it a more efficient and better way to do what you need? I'd be glad to give you some help on this but I need to take
a look at the code and if possible the data collected from a trace (performance counters)

Cheers,

Angel


"Adam Clauss" <cabadam@xxxxxxxxx> wrote in message news:%23u%23Z%23$DNJHA.5232@xxxxxxxxxxxxxxxxxxxxxxx
"Angel Hernandez" <angeljesus14@xxxxxxxxxxx> wrote in message news:4D547513-EC3D-45A3-B274-F30C1C8E8B69@xxxxxxxxxxxxxxxx
Hey mate,

Have you tried setting up some traces and/or performance counters instead of a memory profiler in VS?
What types of counters? We've tried "manual" tracing - pausing in breakpoints at various points of execution to check the sizes of Lists, Dictionaries, etc, nothing unexpected came up.

What does your application do? If you're using non-managed resources, are you disposing them correctly?

We're basically opening all these connections and receiving data. We get the bytes, parse it out (various protocols in use), and periodically (~60 seconds or so typically) perform a set of calculations on the received data. Pretty much everything is straight managed objects (classes which parse out the bytes and expose properties to represent the results, Lists, Dictionaries, etc). As close to an unmanaged resource as we get is the sockets themselves.

There are a somewhat higher number of threads than some of our other applications. Right now, we have a thread-per connection processing data as it gets received off a socket, but those threads tend to spend most of their life in a Monitor.Wait() waiting for data to be put into a queue from the socket code.



We need some more information to provide you some help

Cheers,


--
Angel J. Hernández
MCP,MCAD,MCSD,MCDBA
Microsoft MVP (ASP/ASP.NET)
mailto:angeljesus14@xxxxxxxxxxx

*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
Technical Solution Architect
http://www.customware.net



"Adam Clauss" <cabadam@xxxxxxxxx> wrote in message news:uQR62B%23MJHA.5428@xxxxxxxxxxxxxxxxxxxxxxx
We have an application which is fairly network intensive. During some stress testing, we had it setup to open approximately 300-400 TCP connections (outbound connections, we are a TCP client). Almost immediately upon starting and loading configuration information, the application starts opening up connections and the memory usage immediately skyrockets. With slightly higher numbers of connections, the application actually begins throwing out of memory exceptions; we tuned the number down slightly so we could troubleshoot it without the process crashing.

The strange thing is - only the virtual memory (as described in Task Manager) increases this drastic. After running for 30-45 seconds (and all the connections are open), the Mem Usage (working set) is roughly 200-300mb, which is more or less what we expected. What is unexpected is the virtual memory size, which crosses the 1gb mark.

We attached the memory profiler from Visual Studio, but we simply cannot account for that volume of memory anywhere. If we add up things, we can typically justify the working set, but nothing in the gigabyte quantity.

Any thoughts on what type of things would cause the virtual memory to shoot up so high? Is it some sort of side-effect from having that number of open TCP connections that we are stuck with?

Been banging our head against the wall on how to approach this one, appreciate any comments.

Thanks!

--
Adam Clauss




--
Adam Clauss

.

Relevant Pages

  • Re: ZX Spectrum - cant get it repaired - help!
    ... Swapped ROM (not 100% usre if that one works, ... o All 4116 RAM's are 100% ok; all connections were checked, ... sound A and after a second or so, it makes sound B and it keeps sound ... It`s been a long time, but I used to repair Spectrums regularly, and what you describe is symptomatic of one of the memory ic`s being faulty, have you tried it with the 32k extension removed? ...
    (sci.electronics.repair)
  • Large virtual memory, small working set
    ... During some stress testing, we had it setup to open approximately 300-400 TCP connections. ... Almost immediately upon starting and loading configuration information, the application starts opening up connections and the memory usage immediately skyrockets. ... The strange thing is - only the virtual memory increases this drastic. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: ZONE_NORMAL memory exhausted by 4000 TCP sockets
    ... The problem is the memory. ... concurrent connections, I know the memory size of ZONE_NORMAL would be ... kernel memory allocation might use twice the ram you intend to use because of power of two alignments. ...
    (Linux-Kernel)
  • Re: ZONE_NORMAL memory exhausted by 4000 TCP sockets
    ... > By configuring ebtables and iptables, an application is running as TCP ... > The problem is the memory. ... > concurrent connections, I know the memory size of ZONE_NORMAL would be ... but other things may consume ram on your kernel. ...
    (Linux-Kernel)
  • ZONE_NORMAL memory exhausted by 4000 TCP sockets
    ... an application is running as TCP ... The problem is the memory. ... concurrent connections, I know the memory size of ZONE_NORMAL would be ... is limited by the capability of the client application. ...
    (Linux-Kernel)
Quantcast