Re: Memory limit reached with Windows Mobile

Chris,

Ok, that's a start, but it's only telling you physical memory. Take a look
at available virtual, I think it will be more useful here (it tells you
virtual space available in your slot).

I just run my sample/test application checking AvailableVirtual
instead.

Before loading my DLLs, it returns 28,180,480.
I then load my DLLs until I get a MissingMethodException trying to
load a DLL.
It then returns 15,335,424.

It apparently used 12,85 Mb of Virtual Memory, and although there is
still 15,35 Mb left, it's not possible to load anymore DLLs.
Nevertheless, I can still bring down that AvailableVirtual down to
under 1 Mb by loading DateTime arrays.

To complete the test, I take down the AvailableVirtual to 13 Mb left
by loading DateTime arrays, and then try to load Native DLLs.
I get a MissingMethodException on the very first try.

It would seem that both are allocated in the same virtual memory space
then?
Yet Native DLLs can only be loaded in the "first 12,85 Mb of that
virtual memory"?

In our "real" application, it seems that at some point, we have used
over 12,85 Mb of Virtual Memory, and that we can't load any Native
DLLs after that.
Simply by changing the order in which our application is loaded
(Native DLLs first, then the rest of the application), we managed to
fix 99% of the application.
For some reason though, "something" needs to be loaded when an HTTPS
connection is established, and we can't figure out what or where.

Regards,

--
Patrick

On 27 fév, 14:27, "<ctacke/>" <ctacke[@]opennetcf[dot]com> wrote:
I do feel there is a misunderstanding about our sample/test
application, which is only meant to test the limits and stress the
impact between loading Native DLLs and loading "C# stuff". Your help
is greatly appreciated though!

<ctacke>
No, I think we both understand fully what you're doing. What we're trying
to get at is the "why" of what you're doing. After all running a test just
for the sake of running it isn't useful. You need to know exactly what
you're testing.
</ctacke>

I don't know for sure where everthing is loaded (shared, process
space, ROM XIP space..) since the compact framework is doing that by
himself.

<ctacke>
And therein lies the problem. Once you can determine the load on the
process slot and shared memory, we can figure out what it's running out of
and then treat the "why".
</ctacke>

I use OpenNetCf.Win32.Core.GlobalMemoryStatus().AvailablePhysical to
get an idea of the RAM our application used (by comparing the value
just before and just after).

<ctacke>
Ok, that's a start, but it's only telling you physical memory. Take a look
at available virtual, I think it will be more useful here (it tells you
virtual space available in your slot).
</ctacke>

When you load the interface classes, you're not walking each class and
its
methods and properties via reflection are you? If so that's going to
take
up some serious space.

That's interesting. We use Reflection to call a given method from a
Factory class in our modules. This method returns a class, which
implements a known interface, and we store this class in a Hashtable
for future use. We then call the methods of this class through the
Interface. Could that be eating up space? Is there a better pattern?

<ctacke>
Not necessarily anything better. Using reflection is fine - but when you
load via reflection, a single copy of every item gets loaded into the
AppDomain heap. Inspecting an assembly by walking all of its classes and
methods (like if you were doing something like Reflector) will generate a
lot of these objects. The AppDomain heap can never be shrunk, so if you
load a lot up, you've killed a lot of your process space until the app
exits.
</ctacke>

Entrek is great for native apps, but of no value for managed apps. I'd
look at the CF 1.0 perf counters first, and then watch the app with RPM
and see what's up. Do a web search for RPM or Remote Perfomrnace
Monitor
with the CF. It's likely to show you pretty quickly how things are
going
bad (unfortunately its granularity doesn't yet show the "where" so
that'll
take some detective work).

I tried to use the mscoree.stat file by adding the registry entry, but
the file ended up empty..

<ctacke>
It never gets filled until the app actually exits.
</ctacke>

Do you know any good performance monitor for the CF 1.0?

<ctacke>
Unfortunately no, there isn't one. Running against CF 2.0 and using RPM is
your shortest route to really understanding what's going on here.
</ctacke>

I'll run our application on the CF 2.0 as soon as possible, but that
probably won't be this week...

Thanks again,
Regards,

--
Patrick AVENEL

On 27 fév, 09:07, "The PocketTV Team" <supp...@xxxxxxxxxxxx> wrote:



i completely understand Patrick's points.

apparently ctacke does not understand what a test is and also that
real-life
applications sometimes need many dll's and lots ofmemory.

Patrick, i hope you can find someone who understand what you are
explaining
(i understand all very well), and who knows the answer to your questions
(that, i don't know, but i undersatnd what you are trying to test, and
why).

ctacke: re-read Patrick's questions, maybe you will understand better. he
explained all quite well, and i understand his point, that apparently you
don't get.

"<ctacke/>" <ctacke[@]opennetcf[dot]com> wrote in message

news:ui%23IftdWHHA.1636@xxxxxxxxxxxxxxxxxxxxxxx

This application can either :
1- Load the same DLL (copied with a different name) over and over.
2- Fill an ArrayList with DateTime[1000] while checking free RAM on
the device.

<ctacke>
Checking RAM by what mechanism?
<ctacke>

We noticed that :
- after 60 DLLs loaded with (1), we get a MissingMethodException.

<ctacke>
Sure. Loading native DLLs takes at least 64k each. 60 DLLs would be
~3.8MB of virtual space. How is this valid? Does your app actually
need
60+ native DLLs?
</ctacke>

- after 25 Mb used with (2), we get an OOMException.

<ctacke>
Again, exactly what is this testing? I'm not sure exactly how big a
DateTime is (your number or 1k items indicates 25k which seems really,
really high), but does your app actually contain 25MB of data like this?
If so, this is probably not an ideal design.
</ctacke>

- if we run (1) all the way, we can only load 13 Mb with (2).
- after 12 Mb of (2) we can't load a single DLL with (1).
If we duplicate our test application :
- we can run (2) all the way to 25 Mb each in parallel, for a total of
50 Mb used.
- if we load 20 DLLs with (1), then run (2) all the way (up to ~21
Mb), we can load 40 other DLLs with the second application before
getting a MissingMethodException.

<ctacke>
Again I'm not sure I "get" the point of all of this. First, are you
really loading 60 DLLs? If not, then there's no reason to be testing
that. If you *must* load that many DLLs, are they all out of your
control
or can they be combined? Doug points out that lots of small DLLs is not
a
good thing due to the 64k minimum virtualmemorysize each will take.

Are you really using 25 or even 12MB of arrays of small objects? If so,
this seems like a not-so-wise implementation, but we have no idea on how
you could work around it because the sample/test doesn't seem terribly
valid.
</ctacke>

Based on these measures, we decided to pre-load all our Native DLLs
prior to loading our ".NET Modules". And it works fine for everything
we could pre-load.

Your managed app could be taking up all of the virtual space with the
GCHEap
and preventing native DLLs from loading. I've never seen a managed app
take
up 25, or even 12MB of heap space though. If it is in fact doing that,
I
suspect you're holding some really big items that you shouldn't be, but
I
don't knwo what your "data" is. 25MB of process space usage is huge
for
any
app, managed or otherwise.

We're not at 25 Mb yet, but probably around 14 Mb at worst. Although

<ctacke>
Again, 14MB by what measure? Is it 14MB of process space, or 4MB or
process and 10 or shared? You must be able to answer that to make any
headway on this problem.
</ctacke>

the application is still going to grow with GPRS synchronisation
working in background, etc...

<ctacke>
Not sure how SYnchronization has an affect on it. Are you using
colossal
DataSets that you carry around?
</ctacke>

I know it's a pretty big application, and we tried to reduce what we
could.
We are getting close to having 100 C# DLLs in our application, among
which 40 are interfaces. Most of our data (~200 Mb) is stored in
databases though, and we already cleaned a lot of our data cache.

<ctacke>
Again, managed DLLs have no affect of the process slot virtualmemory-
they are loaded asmemorymapped files from shared space. 100 DLLs or
1000 won't matter.
</ctacke>

We load our ".NET Modules" using reflection, and then store them in a
HashTable for future use, and a lot of these modules are interacting,
and have to be loaded in parallel. Maybe that's part of our problem...

<ctacke>
When you load the interface classes, you're not walking each class and
its
methods and properties via reflection are you? If so that's going to
take
up some serious space.
</ctacke>

You can't "keep" anymemory. The GC handles all allocations and frees
for
you. Again, if you've got 25MB allocated and the GC is unable to
release
any of it because it's all got active roots, there's not much that can
be
done other than change the way your app works.

The GC does release some of thememory, but as long as it doesn't
reduce the allocations under 12 Mb, we are still stuck when trying to
load Native DLLs.

<ctacke>
How much is "some"? It would be well worth building this against 2.0
just
so you can run RPM against it and see what it's doing.
</ctacke>

I'm still unclear what 25Mb of "usage" means in your app. How are you
deriving this nuymber? Used from where? Again, I've never seen an app
using that much virtual space (not one that's behaving well anyway), so
without knowing what's making the allocations or from where, all I can
do
is
guess.

I got the number from our test application, we are not there yet in
our real application.
Part of my problem comes from that our customer wants his application
to be able to read and control RFID cards and 2D barcodes, print
tickets (and barcodes) on a bluetooth printer, use very large
databases to determine the price of the tickets, keep track of the
daily activity of the user, access WebServices and WebSites via GPRS,
synchronise and update the PDA via WiFi or Ethernet, etc. while having
desktop-like performance.

<ctacke>
For the most part this shouldn't be a problem (using the items, not
necessarily achieving desktop speed of course)
</ctacke>

A lot of this requires the use of a few Native DLLs, and we have 70
different screens... so far, we had architectured the application
towards modularity and performance rather

...

plus de détails »- Masquer le texte des messages précédents -

- Afficher le texte des messages précédents -


.

Relevant Pages

  • Re: Memory limit reached with Windows Mobile
    ... look at the CF 1.0 perf counters first, and then watch the app with RPM ... 1- Load the same DLL over and over. ... - after 60 DLLs loaded with, ... Loading native DLLs takes at least 64k each. ...
    (microsoft.public.pocketpc.developer)
  • Re: Memory limit reached with Windows Mobile
    ... Before loading my DLLs, it returns 28,180,480. ... I then load my DLLs until I get a MissingMethodException trying to ... and then try to load Native DLLs. ... good thing due to the 64k minimum virtualmemorysize each will take. ...
    (microsoft.public.pocketpc.developer)
  • Re: Im losing my memory ...
    ... that you dynamically load. ... Your new launcher app would - pre-reserve a large range of contiguous ... I know that the app itself and probably a hundreds of DLLs (maybe even ...
    (microsoft.public.win32.programmer.kernel)
  • Re: windbg & vs 2005 - cant use debugging symbols on vs2005 dlls!
    ... The problem is when we migrated our dlls to ... suddenly their symbols would not load anymore (and they also appear ... > to crash, even though they compile fine, which is why I tried to debug). ... Likewise, when I debug the dlls solution with the app, the app will ...
    (microsoft.public.vc.debugger)
  • Re: Im losing my memory ...
    ... that you dynamically load. ... The launcher app would expose an api to the dll, ... I know that the app itself and probably a hundreds of DLLs (maybe even ...
    (microsoft.public.win32.programmer.kernel)