Re: Inter-process communication

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

On Fri, 06 Apr 2007 10:25:08 -0700, Jon Davis <jon@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

Can you document the claim that "IIS 'self-tunes' itself
and avoids taking up more than roughly one quarter of the available
physical RAM"?


http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/ad56540c-2323-4316-b981-7ebb70352baa.mspx

There is nothing in that document that suggests IIS limits itself to 1/4 of available RAM. As far as "tuning" more generally goes, you'll note (I hope) that part of the tuning that goes on is IIS ensuring that it doesn't starve other processes of memory. Your proposed change to your setup involves essentially doing just that, perhaps even starving IIS.

This means nothing. If you are running 32-bit Windows, with 4GB of RAM it is absolutely normal for an application to not be able to allocate any
more memory even while physical RAM has 1GB or more available.

That's not true; a standalone app can and will allocate RAM and will begin using available virtual memory when unreserved physical RAM is no longer
available.

Wrong. And until you correct your misunderstanding of the memory management model used by Windows, you are going to continue trying to solve your problem the wrong way.

Quick summary (applies to 32-bit Windows, same principles apply -- mostly -- to 64-bit Windows, but of course the actual limits are vastly higher):

* ALL memory allocations in a process go through virtual memory. ALL of them. A normal application (this would include your service) under Windows does not get to allocate physical RAM directly. The first part of any memory allocation involves allocating a chunk of virtual address space, whether or not there is sufficient physical RAM to satisfy that allocation.

* The virtual address space for 32-bit pointers is theoretically 4GB, but because of the way that Windows uses those pointers, only half of this is available to the process directly (up to 3GB with a special switch booting the OS). Thus, no process can EVER allocate more than 2GB. Because of fragmentation, most processes will find themselves limited to maximum total allocations somewhat less than 2GB. The larger the individual blocks the process is trying to allocate, the more likely this will be a problem.

* When a process references a virtual address that hasn't been "committed", at that point physical RAM is assigned to the virtual address. But since a process cannot address more than 2GB of virtual address space, the process can never reference more than 2GB of physical RAM as well. In reality, a process almost never has their entire virtual allocation committed, and so physical RAM usage is almost always less (and usually considerably less) than their total virtual memory allocation.

IIS is FAR more likely to throw OutOfMemoryExceptions than a
standalone app. I may not be a world-class developer yet but 10 years of
experience with developing both IIS apps and Windows apps gives me enough to go on.

It's apparent from your lack of understanding of the Windows memory management model that that's not true. You need to accumulate at least a little more experience before you have "enough to go on".

IIS memory utilization in itself is not the issue here.

Then you have asked your question incorrectly. In all of your posts, how IIS uses memory is a central part of your complaint. If IIS memory utilization is not the issue, then I recommend you stop including it in your queries.

The issue is we have a process that we have a large process that we've decided will be
implemented as a Windows service and that will be accessible by multiple web apps.

As near as I can tell, the reason you've decided to implement it as a Windows service is that you believe you can get around the memory allocation issues by doing so. The problem is that you can't. The same memory allocation issues that restrict IIS will also restrict you.

Pete
.

Relevant Pages

  • Re: Setting VM with WinXP-2K_Pagefile.vbs
    ... Also this is on my Backup XP where I have no RAM hogs ... >because of the unused portions of memory allocation requests. ...
    (microsoft.public.windowsxp.general)
  • Re: Dictionary Throws OutOfMemoryException
    ... | application has only allocated 1.3GB of RAM (according to the Task ... | memory is set to an initial 2GB with a maximum of 4GB of RAM. ... that you should initialize the dictionary to the size (in entries) you are ... and restrict the allocation of entries to what you effective need (well, ...
    (microsoft.public.dotnet.languages.csharp)
  • How to guarantee that an embedded (RealTime) application never causes a page fault
    ... What solutions exist for locking an application in RAM that uses dynamic memory ... quite a lot of dynamic memory allocation and deallocation takes ... heap that is first made resident in RAM. ...
    (comp.os.linux.development.system)
  • Re: Allocating kernel memory
    ... > I never complained about the real memory that was used but pointed out ... > that a large portion of the kernel virtual address space is used up. ... RAM you should avoid PAE, ... But it was limited to 1GB of physical RAM. ...
    (comp.os.linux.development.system)
  • Re: Active Memory Defragmentation: Our implementation & problems
    ... > Would memory fragmentation have any appreciable impact on L2 cache line ... The rest of the hardware makes this look like a bunch of RAM ... address-space. ... that there are few nice clean cache-line fills for physical RAM ...
    (Linux-Kernel)