Re: MALLOC

From: Joseph M. Newcomer (newcomer_at_flounder.com)
Date: 05/30/04 

Date: Sun, 30 May 2004 19:26:04 -0400

Paging isn' treally a good model, for the reason you point out. There can be no
:"lookahead" in paging, whereas if you use ordinary file I/O there CAN be lookahead. I
think you are trying to solve the wrong problem with an overly complex mechanism, and
trying to write a paging mechanism seems unnecessary given that ordinary sequential file
I/O would suffice (we use multimedia I/O calls, which work on structured files, and can
keep up indefinitely with CD-quality audio)
                                        joe

On Sun, 30 May 2004 10:47:12 -0700, "Alexander Grigoriev" <alegr@earthlink.net> wrote:

>Another possible implementation is to use your own paging mechanism, with
>larger granularity and guaranteed contiguosity (?) of the paging file. It
>takes a VirtualAlloc of uncommitted memory and you should handle all
>exceptions on access to uncommitted memory.
>
>Windows paging is kind of crappy, it seems to use 4KB pagefile granularity
>and doesn't seem to do read-ahead for that reason. The better approach would
>be to allocate pagefile space in 64 KB chunks, and page in all non-present
>pages in a 64 KB chunk, when a page fault occurs inside it. I also cannot
>understand why (with the modern RAM sizes) it doesn't read a whole EXE/DLL
>into memory on LoadLibrary or process startup, it would speed up the startup
>greatly.
>
>Of course, the OP problem doesn't need such large allocation, anyway.
>
>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>news:uehib0pvhdbq54v4esv9sj1u2oa4ja10aj@4ax.com...
>> First, dream on. The chances that you will be able to allocate 1GB of
>memory is
>> vanishingly small, even if you call VirtualAlloc.
>>
>> Physical RAM size has nothing to do with the ability to allocate the
>memory. You need a
>> swapfile of a minimum of 1GB, and I would suggest 1.5GB to 2GB But you
>need a whopping lot
>> of main memory to get respectable performance.because if memory is low,
>you spend all your
>> time swapping pages in and out.
>>
>> Why do you need 1GB? There are often better alternatives to such massive
>allocations. For
>> example memory-mapped files where you map a view to pieces of the file.
>>
>> But while you see 2GB of virtual address space, a lot of that is already
>committed to
>> various DLLs, your heap, etc., and finding 1GB of contiguous heap is very
>low probability.
>> On end-user machines, you can assume the probability is zero.
>>
>> You are more likely to be able to allocate two 0.5GB chunks, or 10 100MB
>chunks.
>>
>> If you're seriious about needing this much memory, buy an AMD64 box, which
>has an 8TB user
>> address space.
>> joe
>>
>> On Sat, 29 May 2004 07:41:03 -0700, "VIJAY"
><anonymous@discussions.microsoft.com> wrote:
>>
>> >I need to allocate 1 gb of data in memory.( ram).. how do i do this??
>> >but if the physical ram size is small and how the data is loaded..
>> >any suggestion will be appreciated..
>> > VIJAY
>>
>> Joseph M. Newcomer [MVP]
>> email: newcomer@flounder.com
>> Web: http://www.flounder.com
>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>

Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Relevant Pages

  • Re: running Linux with no swap space (but lots of RAM)
    ... | capable of tolerating paging I/O, not that the system is paging to ... Please explain what you mean by "tolerating paging I/O". ... to swap anything. ... The amount of memory allocated to ...
    (comp.os.linux.development.system)
  • Re: Problems reclaiming VM cache = XFree86 startup annoyance
    ... :I'm not sure what is overloading the disk other than the large amounts ... :of paging that are initiated the moment I invoke "startx". ... :allocable memory), with the subtle distinction being that "Cache" ... :pages used to hold disk blocks now discarded, then why not allocate ...
    (freebsd-stable)
  • Re: MALLOC
    ... larger granularity and guaranteed contiguosity of the paging file. ... exceptions on access to uncommitted memory. ... be to allocate pagefile space in 64 KB chunks, ... even if you call VirtualAlloc. ...
    (microsoft.public.vc.mfc)
  • Perofmance / Paging Question
    ... be a large number of page faults: ... I/O Reads: 11K ... So, why all the paging? ... to tune the system to allow this application to take more memory? ...
    (microsoft.public.win2000.general)
  • Re: Pressuring GC to give up clean up memory.
    ... One of them, it seems, is global system paging activity, and also your processes generated paging. ... Even if you allocate a lot, but the system has no activity, GC doesn't see any reason to release something. ... Several people stated that GC won't clean up memory unless pressured to do so. ... So I wrote a quickie app to gobble up memory 100mb at a time, but I still don't see anything giving up memory. ...
    (microsoft.public.dotnet.languages.csharp)