Re: High-performance IO

Hi Anton,

First of all, memory does not have to be physically contigious - the
only thing you need is to make it resident and locked

Surely, but discontinuities limit the length of the DMA transfers.
If the length of a contiguous memory block is equal to the size
of a virtual memory page, the DMA transfer length will be at
most 4KiB too -- not a very impressive value... The other limiting
factor if the cluster size, so the buffer doesn't have to be contiguous
that much.

But I don't know whether the disk drivers are supposed to support 64-bit
(36, in fact) physical addressing.

Drivers are not concerned about things like that - after all, they deal
only with linear addresses.

But eventually they must issue a DMA transfer request (does
anyone stil use PIO?) and they work only with physical memory.
That rises two questions:

a) what is the upper limit of a DMA disk transfer, how is it
computed on a Windows XP-based machine and by whom?

b) is the underlying hardware able to access physical memory
above 4GiB? There may still be some 32-bit oddities etc.

I know that AWE can give the programmer a way to use
huge memory, but the problem is whether a physical memory
block from the AWE area is guaranteed to be reachable via
PCI etc. If one completes a data block in AWE and issue an IO request, it would not be desirable to receive an error
code like E_INVALID_PHYSICAL_MEMORY_RANGE
or something. I ask, because the following excerpt from MSDN
is the seed of doubt:

"A similar restriction is that AWE window address ranges and memory
pools cannot be used as data buffers for graphics or video calls."

And since AGP and PCI buses are being implemented in a quite
similar way, this remark naturally scales up the problem to a question
"does this restriction apply to disks too?"

Unfortunately, there is not much information available at the
level of detail I would like to know... :-(

The only problem is that AWE requires SeLockMemory privilege

I believe it will not be a big problem, especially that it is just an
alternative control flow path -- without that privilege my application
will simply use the old good VirtualAlloc-based way. But indeed,
the temptation to use AWE is very strong... :-)

Best regards
Piotr Wyderski

PS. The current, improved implementation is damn fast
and no longer is the performance limiting factor. :-)

.

Relevant Pages

  • relic Have you learned how to post yet?
    ... As you have a selective memory, here is the source code for you. ... applications with a 4 GB virtual address space. ... that run on computers with more than 2 GB of physical memory. ... Address Windowing Extensions (AWE) enables applications to address more = ...
    (alt.os.windows-xp)
  • Re: Windows 2000 Server + SQL E.Edition - Can it utilize 4GB Memory
    ... You can use the AWE enabled option with SS Enterprise Edition or Developer ... There is memory below the 4 GB, ... must swap back and forth between this space and the USER MODE memory space ... If you have just plain Win2K Server and not advanced Server you can not use the /3GB switch. ...
    (microsoft.public.sqlserver.server)
  • Re: Windows 2000 Server + SQL E.Edition - Can it utilize 4GB Memory
    ... You need to set the max memory due to the fact that when you set AWE the ... Meaning that SQL Server will grab all that it ... using ADV Server and forgetting all about AWE if all you have is 4GB. ...
    (microsoft.public.sqlserver.server)
  • Re: Allocating more memory to sql server
    ... Are you sure the AWE was set correctly? ... > for SQL server Targer Sever memory and Total Server Memory, ... YOu should not use Task Manager to monitor memory for>> SQL Server. ... >>> windows task manager total physical memory shows as memory usage ...
    (microsoft.public.sqlserver.setup)
  • Re: SQL Server user 2GB> ram
    ... I have enable the AWE using the following scripts: ... when I monitor the SQL Server Performance Monitor Total Server ... Memory counter. ...
    (microsoft.public.sqlserver.setup)