Re: calloc ( ) failed ... but enough RAM ...

From: xavier (cose_at_cose.fr)
Date: 02/17/05


Date: Thu, 17 Feb 2005 14:07:24 +0100
To: "Joseph M. Newcomer" <newcomer@flounder.com>


it is true. memory fragmentation is certainly the problem.

I know how many RAM available has my system by simply looking into the
Windows 2000 Task Manager.

There probably is an error in your compute: Try to call calloc( ) for
400 Moctets , it about instantaneously works fine ... are you sure
talking about "ms" ( milliseconds) for a page fault ?
If you are right, it would mean that allocate 10 Mo ( 800 Mo / 80 ... )
would take at best 7mn/80 = 5 seconds ... ?

The solution is - as said Stephen Kellett - to cut the computation and
writing of my datas ...

Thanks and Regards ,
Xavier

Joseph M. Newcomer a écrit:
> And what,exactly, makes you think you "have enough RAM free"? You just asked for
> 800*1024*1024, or 800 Megabytes of memory. It is unlikely in the extreme that you would
> have a slot in your address space large enough to hold that. You are talking about
> keeping 200 or 300MB free, but you have no basis for making that statement.
>
> You are ignoring issues like address space fragmentation. This is significant. You can
> safely assume that allocating a contiguous chunk of memory this size is unlikely.
>
> Is your swap space large enough to handle an allocation this size?
>
> BY the way, why "calloc"? This has to zero the memory, which means it has to touch each of
> the (1024x1024x800)/4096 pages, or 204,800 pages you are allocating. That will generate
> 204,800 page faults.Assuming 20ms per page fault, this will take 4,096 seconds to execute,
> or a little over an hour (68 minutes). At 10ms per page fault, it will take a half hour.
> Being really optimistic and saying 2ms per page fault, it will take about seven minutes to
> zero this memory out. There's something deeply wrong with what you are trying to do.
>
> There is no way to answer your question about how much space you can allocate; you can't
> answer this until you allocate it, because previous allocation patterns have also
> fragmented memory. So you have both heap fragmentation and address space fragmentation
> coming into play here.
>
> What are you really trying to do?
> joe
>
> On Wed, 16 Feb 2005 12:29:42 +0100, xavier <cose@cose.fr> wrote:
>
>
>>Hello to all
>>
>>I need to allocate a big buffer but calloc failed , even if I do have
>>enough RAM free.
>>I have 1200 Mo free in RAM when calloc( ) is called , but it works only
>> if I calloc 900 Mo in my application.
>>
>>
>>
>>see the code below: it try to allocate the max RAM memory
>>
>>
>>"
>>unsigned short *tabtest = NULL ;
>>int memtaille = 800*1024 ;
>>int mempas = 1024 ;
>>while( (tabtest == NULL) && (memtaille>mempas) )
>>{
>> memtaille -= mempas ;
>> tabtest = (unsigned short *)calloc( memtaille*1024 , sizeof(unsigned
>>short) );
>>}
>>if(tabtest != NULL)
>>{
>> free(tabtest);
>> tabtest = NULL ;
>>}
>>"
>>
>>in a new MFC project , this code can allocate all the free (available)
>>memory in RAM.
>>
>>in my application , depending on where in the code I put this code, it
>>will always keep 200 or 300 Mo free in RAM.
>>
>>why ? I don t think I have memory leaks ( no bugs ... no debug messages
>> when application exits ... )
>>how to allocate all the RAM available ?
>>how to know exactly how much RAM I can allocate with calloc() before
>>calling it ?
>>
>>
>>Thanks.
>>Xavier
>>
>>
>
>
> Joseph M. Newcomer [MVP]
> email: newcomer@flounder.com
> Web: http://www.flounder.com
> MVP Tips: http://www.flounder.com/mvp_tips.htm



Relevant Pages

  • Re: Memory problems (possibly very easy question)
    ... Now allocate 25,000 objects. ... Once you get the leaks fixed, fragmentation becomes your worst enemy. ... memory usage warnings. ... But releasing it does not reduce the footprint, it merely keeps it from growing as the ...
    (microsoft.public.vc.mfc)
  • Re: Memory fragmentation issue in kernel mode
    ... Also, on the Performance tab, watch the Kernel Memory section. ... fragmentation, which is actually fragmentation of the system virtual ... A user level application doesn't allocate and even access the ... I have my driver that uses direct IO to perform data transfer ...
    (microsoft.public.development.device.drivers)
  • Re: Page File size wont change
    ... PF on my USB disk in an effort to prevent fragmentation of the PF. ... the system disk of 1.5 times the RAM size. ... Putting the pagefile on it's own partition even better. ... memory that is required. ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: [RFC] Make balance_dirty_pages zone aware (1/2)
    ... >> But if the system has been running for a while, memory is full of pagecache, ... We try to allocate from the local node, fail, and fall back to the ... so there's < 40% of RAM for each node). ... > to not use lowmem for pagecache. ...
    (Linux-Kernel)
  • Re: out of memory errors & Pocket PC program vs. storage RAM allocatio
    ... >Is there a trick to to getting Pocket PC 2002 to allocate more RAM for an ... >mode, 275k in release mode, and uses ~1MB of RAM when running. ... it occassionally posts an out of memory dialog during parsing. ...
    (microsoft.public.pocketpc.developer)

Loading