Re: Memory limit reached with Windows Mobile

Chris,

The GCHeap is allocated out of the 32MB process slow, so it is bounded by
available memory, but oinbounded by the CF itself (unlike in 1.0). You
"loaded" 25Mb how? Where? how do you know it's all in the GCHeap (did you
run RPM to see?).

We are actually using the CF 1.0, although we didn't notice any
difference with the CF 2.0.
Our application was getting more and more "random"
MissingMethodException while loading Native DLLs when most of our
".NET modules" (RFID, IR Scanner, Bluetooth printing, GPRS, WiFi, SQL
merge, Reporting, Ticketing, etc.) had been loaded. We soon determined
that it was coming from a lack of memory...

To better understand the way it all works, we created a test
application.

This application can either :
1- Load the same DLL (copied with a different name) over and over.
2- Fill an ArrayList with DateTime[1000] while checking free RAM on
the device.

We noticed that :
- after 60 DLLs loaded with (1), we get a MissingMethodException.
- after 25 Mb used with (2), we get an OOMException.
- if we run (1) all the way, we can only load 13 Mb with (2).
- after 12 Mb of (2) we can't load a single DLL with (1).
If we duplicate our test application :
- we can run (2) all the way to 25 Mb each in parallel, for a total of
50 Mb used.
- if we load 20 DLLs with (1), then run (2) all the way (up to ~21
Mb), we can load 40 other DLLs with the second application before
getting a MissingMethodException.

Based on these measures, we decided to pre-load all our Native DLLs
prior to loading our ".NET Modules". And it works fine for everything
we could pre-load.

Your managed app could be taking up all of the virtual space with the GCHEap
and preventing native DLLs from loading. I've never seen a managed app take
up 25, or even 12MB of heap space though. If it is in fact doing that, I
suspect you're holding some really big items that you shouldn't be, but I
don't knwo what your "data" is. 25MB of process space usage is huge for any
app, managed or otherwise.

We're not at 25 Mb yet, but probably around 14 Mb at worst. Although
the application is still going to grow with GPRS synchronisation
working in background, etc...
I know it's a pretty big application, and we tried to reduce what we
could.
We are getting close to having 100 C# DLLs in our application, among
which 40 are interfaces. Most of our data (~200 Mb) is stored in
databases though, and we already cleaned a lot of our data cache.
We load our ".NET Modules" using reflection, and then store them in a
HashTable for future use, and a lot of these modules are interacting,
and have to be loaded in parallel. Maybe that's part of our problem...

You can't "keep" any memory. The GC handles all allocations and frees for
you. Again, if you've got 25MB allocated and the GC is unable to release
any of it because it's all got active roots, there's not much that can be
done other than change the way your app works.

The GC does release some of the memory, but as long as it doesn't
reduce the allocations under 12 Mb, we are still stuck when trying to
load Native DLLs.

I'm still unclear what 25Mb of "usage" means in your app. How are you
deriving this nuymber? Used from where? Again, I've never seen an app
using that much virtual space (not one that's behaving well anyway), so
without knowing what's making the allocations or from where, all I can do is
guess.

I got the number from our test application, we are not there yet in
our real application.
Part of my problem comes from that our customer wants his application
to be able to read and control RFID cards and 2D barcodes, print
tickets (and barcodes) on a bluetooth printer, use very large
databases to determine the price of the tickets, keep track of the
daily activity of the user, access WebServices and WebSites via GPRS,
synchronise and update the PDA via WiFi or Ethernet, etc. while having
desktop-like performance.
A lot of this requires the use of a few Native DLLs, and we have 70
different screens... so far, we had architectured the application
towards modularity and performance rather than memory usage, although
we did try to reduce that lately. The user constantly switches between
the different modules, and loading them seems pretty long (even if it
only takes a few seconds), and could be a reason for them not to use
the application anymore if we unloaded them everytime (we do unload
the screens though, but usually not the business parts and Native
DLLs). The technical or transverse modules also have to be loaded at
all times since they are used by most functions.

I'm afraid we will have to rethink part of the application to reduce
the memory usage, but as I said, it's frustrating when it's so close
to working perfectly that way (by pre-loading Native stuff before
loading our different modules).
Our test application seemed to indicate that we still had a good
margin (up to 25 Mo) before having OOM issues.
The MissingMethodException issue after 12 Mb is disturbing though.

did you run RPM to see?

I haven't heard of RPM yet. I had looked into Entrek a bit, but
haven't used it yet.
Is there any chance of finding what is loaded during the first HTTPS
connection with such a tool? I still believe that would be our easy
way out, since reachitecturing our application will not be an easy
thing to do right now.

Thanks a lot for your quick answers,
Regards,

--
Patrick AVENEL

On 26 fév, 17:46, "<ctacke/>" <ctacke[@]opennetcf[dot]com> wrote:
1. You said the GC Heap size is unbound, yet we manage to load 25 Mb
of data in one Process before getting an OOM Exception, and still load
another 25 Mb of data in another process without unloading the data
from the first process. (The system free memory is down 50 Mb). That
doesn't fit with the way I understand the way shared memory works.

<ctacke>
The GCHeap is allocated out of the 32MB process slow, so it is bounded by
available memory, but oinbounded by the CF itself (unlike in 1.0). You
"loaded" 25Mb how? Where? how do you know it's all in the GCHeap (did you
run RPM to see?).
</ctacke>

2. You talk about "Native stuff" at the beginning of your
presentation, but I didn't understand where these Native DLLs get
loaded.

<ctacke>
Native DLLs follow the normal model and load into slot 0, like the article
you referenced shows
<c/tacke>

Our problem comes from the fact that our CF program seems to
be taking memory space which could otherwise be used to load Native
DLLs. Although we can use up to 25 Mb for our application, once the
application uses over 12 Mb, it's no longer possible to load Native
DLLs in the same process (yet we can load Native DLLs in another
process, as long as the total doesn't exceed 12 Mb of Native DLLs
loaded).

<ctacke>
Your managed app could be taking up all of the virtual space with the GCHEap
and preventing native DLLs from loading. I've never seen a managed app take
up 25, or even 12MB of heap space though. If it is in fact doing that, I
suspect you're holding some really big items that you shouldn't be, but I
don't knwo what your "data" is. 25MB of process space usage is huge for any
app, managed or otherwise.
</ctacke>

I though it would be possible to keep some critical memory space in
the 12Mb range of our process, and free it when we would need to load
new DLLs, but the part about GC Compaction ruined my hopes there : it
seems the memory freed by the CF will always be on top.

<ctacke>
You can't "keep" any memory. The GC handles all allocations and frees for
you. Again, if you've got 25MB allocated and the GC is unable to release
any of it because it's all got active roots, there's not much that can be
done other than change the way your app works.
</ctacke>

It's pretty frustrating that 99% of our application works fine up to
20-25 Mb of memory usage (by loading all the "Native stuff" before
loading the heavy parts of our application), and we might have to
rearchitecture it completely to remain under 12 Mb of memory usage at
all times (probably at a great performance cost) just because we can't
find a way to load whatever is required to establish an HTTPS
connection before the connection is actually established.

<ctacke>
I'm still unclear what 25Mb of "usage" means in your app. How are you
deriving this nuymber? Used from where? Again, I've never seen an app
using that much virtual space (not one that's behaving well anyway), so
without knowing what's making the allocations or from where, all I can do is
guess.
</ctacke>

Thanks again for your help!

Regards,

--
Patrick AVENEL

On 26 fév, 15:03, "<ctacke/>" <ctacke[@]opennetcf[dot]com> wrote:



The Compact Framework doesn't follow the model you're looking at. Its
DLLs
load into sharedmemoryas amemory-mapped file and have no impact on the
32MB process space. The only thing that uses substantial process space is
the GCHeap, and if you're hitting 12MB of heap usage, you might consider
rearchitecting how the app works.

Here's a webcast that might clarify things.

http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=10....

--
Chris Tacke - Embedded MVP
OpenNETCF Consulting
Managed Code in the Embedded Worldwww.opennetcf.com
--

"Patrick A." <patrick.ave...@xxxxxxxxx> wrote in message

news:1172479898.418334.54450@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Hello,

We are currently havingmemoryissues with our Pocket PC application.
We are using PsionTeklogix WorkAbout Pro devices withWindowsMobile
2003 and the Compact Framework 1.0 SP3.
The device has 95.80 Mb of RAM, 30 Mb of ROM and a 512 Mb SD-Card.
Most of our application is loaded directly from the SD-Card.

The limitations we have experienced are the following:
- 25 Mb per process
- 12 Mb for the DLLs (independently from the number of
processes)
- DLLs can only be loaded in the "first" 12 Mb of each
process

For example:
1. If a process loads 4 Mb of DLLs, then 8 Mb of data, he can no
longer load DLLs, but can still load data. A second process launched
in parallel will be able to load a maximum 8 Mb of new DLLs, then
data.
2. If a process loads 12 Mb of data, it can no longer load a single
DLL.

What happens in our application is that at some point we reach the 12
Mblimitand every attempt at loading a library after that fails.

Such behaviour is described here:
http://msdn2.microsoft.com/en-us/library/ms836325.aspx
"Now consider what to do if you encounter a potential problem.
Suppose
Process 2 loaded DLL C that was quite large as shown in Figure 6.
Note
that process 3 has the bad luck of being both a large .exe file and
loading a DLL after process 2 had loaded its rather immense DLL C.
Clearly, process 3 is close to trouble if it attempted to load any
more DLLs that had not been loaded already by other processes. This
is
a somewhat contrived example because the size of DLL C would have to
be incredibly large, or Process 2 would have to load a large number
of
DLLs for this problem to occur naturally."

We do have a large number of DLLs, some of which are quite large and
the previous example is no longer contrived in our case.

To avoid the problem, we tried several approaches:
- Free somememoryin the critical 12 Mb area when we need to load
a new DLL
o The compact framework doesn't seem to free thememoryfor
native DLLs, but only for its own future use.
o We tried to artificially create 12 Mb of data before
loading our real data, and then free the 12 Mb of artificial data to
restore some freememoryin the criticalmemoryspace, but no new DLL
could be loaded afterwards.
o We even tried to broadcast a WM_HIBERNATE message to
force the Compact Framework to free somememory, but in vain.

- Migrate our application toWindowsMobile2005 and the Compact
Framework 2.0
o We had the samememoryissues.

- Split our application in several processes.
o We still reach the 12 Mblimitfor the DLLs, no matter
how many processes are involved.
o Loading our DLLs in a separate process doesn't change
the
behaviour.

- Load all our DLLs before loading data.
o This approach has been quite successful so far.
o We still have some issues with several functionalities
such as HTTPS WebService calls, which fail if the very first call is
made when thememoryhas already passed the 12 Mblimit:
"System.Net.WebException - Could not establish secure channel for
SSL/
TLS".

Is there something to pre-load so that our HTTPS call will already
have its requirements loaded before thememoryhas passed the 12 Mb
limit?

Did we miss something in our understanding of howmemoryis managed
in
WindowsMobile2003?

Our application requires more than 12 Mb ofmemory(most of which is
data and C# .Net libraries), and needs to be able to load new DLLs at
all time, what would
you recommend in this situation?

Thank you in advance,

Sincerely,

--
Patrick AVENEL- Masquer le texte des messages précédents -

- Afficher le texte des messages précédents -- Masquer le texte des messages précédents -

- Afficher le texte des messages précédents -


.

Relevant Pages

  • Re: Memory limit reached with Windows Mobile
    ... That would explain the memory problem - you just can't do that. ... Actually, simply by loading our 18 Native DLLs, we reduce the Virtual ... able to load any more Native DLLs. ...
    (microsoft.public.pocketpc.developer)
  • Re: Memory limit reached with Windows Mobile
    ... That would explain the memory problem - you just can't do that. ... You have to load stuff when it's ... All native DLLs get loaded ...
    (microsoft.public.pocketpc.developer)
  • Re: Memory limit reached with Windows Mobile
    ... I do understand what a test is, and I do understand how memory management ... ctacke: re-read Patrick's questions, ... 1- Load the same DLL over and over. ... Loading native DLLs takes at least 64k each. ...
    (microsoft.public.pocketpc.developer)
  • Re: Memory limit reached with Windows Mobile
    ... You said the GC Heap size is unbound, yet we manage to load 25 Mb ... (The system free memory is down 50 Mb). ... but I didn't understand where these Native DLLs get ... DLLs in the same process (yet we can load Native DLLs in another ...
    (microsoft.public.pocketpc.developer)
  • Re: Memory limit reached with Windows Mobile
    ... That would explain the memory problem - you just can't do that. ... Once I reached the 12 Mb limit, I cannot load any DLLs, yet if I ... Actually, simply by loading our 18 Native DLLs, we reduce the Virtual ...
    (microsoft.public.pocketpc.developer)