Re: W3WP memory usage

From: Kevin Spencer (kevin_at_takempis.com)
Date: 03/11/04 

Date: Thu, 11 Mar 2004 16:26:33 -0500

Hi Paul,

You may be correct regarding the libraries using the lion's share of memory.
I am not familiar with the inner workings of these libraries. However, I do
know that the solution I offered you will work. I'm not sure what you're
using SharePoint for, so I can't advise you as to an alternative to
SharePoint. 

-- 
HTH,
Kevin Spencer
.Net Developer
Microsoft MVP
Big things are made up
of lots of little things.
"Paul" <a@b.com> wrote in message
news:uYPdE06BEHA.3284@TK2MSFTNGP09.phx.gbl...
> Kevin, thank you for your response.
>
> I am quite aware that Sharepoint files are stored in the database.  When I
> said "use the file system", I meant that I will not store the files in
> Sharepoint at all, but instead store them in the file system, effectively
> giving up on Sharepoint altogether.
>
> When you say "whole lot of hits to that database", I assume you mean the
set
> of queries the sharepoint classes send to the SQL db to request the site
> names and document names.  My experience is with SQL server, and I can
> assure you that querying a couple dozen rows from it is NOT causing a 50M
> memory request.  Besides, the memory usage is in W3WP, and not SQL Server.
> The SQL server process never goes over 20Meg of usage!
>
> No, the Sharepoint libraries are hogging the memory, and for some reason
are
> not releasing it, and then dying under their own weight (bringing IIS with
> it).
>
> Now, if you are telling me that the SPFile class, when instanciated,
stores
> the entire file in memory, then I could believe that it might need 10 or
so
> meg on a request (the complete size of all the files I am storing in
> sharepoint at this time).  But nothing in the (hopeless) Sharepoint
> documentation would indicate that.  I am simply requesting the names and
> urls to the different files.
>
> And so, my question is basically the same: What is causing the Sharepoint
> Class Library to eat 50M of memory on a request of what is approximately
20K
> worth of data?
>
>
> "Kevin Spencer" <kevin@takempis.com> wrote in message
> news:OtGFVQ6BEHA.3404@TK2MSFTNGP10.phx.gbl...
> > You have a bigger problem than you realize. Check your file system. The
> > pages aren't there. They are all stored in the database, and your app is
> > making a whole lot of hits to that database, which is the reason for all
> the
> > excess overhead. So, you won't be able to use the file system to
enumerate
> > sites/pages and/or gather information about them. Your best bet would be
> to
> > run this utility as a Service and store the data in a database. The
> Service
> > only has to "catalogue" the content maybe once a day, and your Web
Service
> > can hit the database to get the "catalogue" of pages.
> >
> > -- 
> > HTH,
> > Kevin Spencer
> > .Net Developer
> > Microsoft MVP
> > Big things are made up
> > of lots of little things.
> >
> > "Paul" <a@b.com> wrote in message
> > news:#BTs3U5BEHA.2804@tk2msftngp13.phx.gbl...
> > > I wrote a web service that uses the Sharepoint Class Library to
navigate
> a
> > > sharepoint site collecting the names of all the Webs, their shared
> > > documents, and the versions, and returns it to a client.  Memory usage
> of
> > > the w3wp process goes insane, grabbing 50 Meg chunks just about every
> time
> > I
> > > traverse the tree (but sometimes only 5M).  Sometimes the w3wp process
> > gives
> > > the memory back, and other times it heads on up to around 700Meg, but
it
> > > always eventually craps out, at which point I get out of  memory
errors
> on
> > > the server (on any IIS web site defined) and I have to stop and start
> the
> > > world wide web publishing service.  Some web sites return a page with
> > > "-2146893056 (0x80090300)" on them, and nothing else.
> > >
> > > The code is fairly simple (just loop through each web and collect its
> > Shared
> > > Documents files and their version's names, and recursively call for
sub
> > > webs).  The whole collection is pretty small, about 10 sites, with a
few
> > > nested sites no more than 3 deep.  I would guess the resulting data
> > > structure that gets returned through the web service is no more than
20K
> > or
> > > so.  My server is running SBS2003 and has a Gig of RAM.
> > >
> > > I am wondering if there is a resource that I am not closing properly
and
> > > somehow all the sharepoint libraries are getting loaded every time.
If
> > > there is no fix, We're going to have to chuck out Sharepoint
altogether,
> > and
> > > just use the File System.
> > >
> > > Here is the code, in case you are interested.  Sorry about the
tabbing.
> > Any
> > > help would be appreciated. Thanks, Paul
> > >
> > > public class SPWSTree
> > > {
> > > public static SPTree GetTree(string i_server, string i_site, int
> i_depth)
> > > {
> > > if (i_depth > 5)
> > > return new SPTree();
> > > // create the new Tree Node
> > > SPTree a_returnNode = new SPTree();
> > > // get a handle on the site.
> > > SPSite a_site = new SPSite(i_server);
> > > if (i_site == "/")
> > > {
> > > a_returnNode.m_url = "";
> > > a_returnNode.m_displayName = "XESP";
> > > }
> > > else
> > > {
> > > a_returnNode.m_url = i_site.Substring(i_site.LastIndexOf("/"));
> > > a_returnNode.m_displayName = a_site.AllWebs[i_site].Title;
> > > }
> > > ArrayList a_folders = new ArrayList();
> > > foreach (SPWeb a_web in
> a_site.AllWebs[i_site].GetSubwebsForCurrentUser())
> > > {
> > > a_folders.Add(GetTree(i_server, a_web.ServerRelativeUrl, i_depth +
1));
> > > }
> > > if (a_folders.Count > 0)
> > > a_returnNode.m_folders = (SPTree [])a_folders.ToArray(typeof(SPTree));
> > > // now get all the files in this site's shared documents folder
> > > SPFileCollection a_fileCollection = null;
> > > try
> > > {
> > > SPFolder a_sharedDocuments = a_site.AllWebs[i_site].GetFolder("Shared
> > > Documents");
> > > a_fileCollection = a_sharedDocuments.Files;
> > > }
> > > catch (Exception /* Ex */)
> > > {
> > > // a_returnNode.m_displayName = "1:" + Ex.Message;
> > > }
> > > if (a_fileCollection != null)
> > > {
> > > ArrayList a_files = new ArrayList();
> > > foreach (SPFile a_file in a_fileCollection)
> > > {
> > > SPTree a_fileNode = new SPTree();
> > > a_fileNode.m_url = "/" + a_file.Url;
> > > a_fileNode.m_displayName =
> > a_file.Url.Substring(a_file.Url.LastIndexOf("/")
> > > + 1);
> > > if (a_file.CheckedOutBy != null)
> > > a_fileNode.m_checkedOutBy = a_file.CheckedOutBy.LoginName;
> > > a_fileNode.m_currentVersion = a_file.UIVersion.ToString();
> > > ArrayList a_versions = new ArrayList();
> > > foreach (SPFileVersion a_fileVersion in a_file.Versions)
> > > {
> > > SPVersion a_version = new SPVersion();
> > > a_version.m_comment = a_fileVersion.CheckInComment;
> > > a_version.m_createdBy = a_fileVersion.CreatedBy.LoginName;
> > > a_version.m_createdOn = a_fileVersion.Created.ToString("r");
> > > a_version.m_size = a_fileVersion.Size.ToString();
> > > a_version.m_id = a_fileVersion.ID.ToString();
> > > a_versions.Add(a_version);
> > > }
> > > SPVersion a_currentVersion = new SPVersion();
> > > a_currentVersion.m_comment = a_file.CheckInComment;
> > > a_currentVersion.m_createdBy = a_file.ModifiedBy.LoginName;
> > > a_currentVersion.m_createdOn = a_file.TimeLastModified.ToString("r");
> > > a_currentVersion.m_size = a_file.Length.ToString();
> > > a_currentVersion.m_id = a_file.UIVersion.ToString();
> > > a_versions.Add(a_currentVersion);
> > > a_fileNode.m_versions = (SPVersion
> > [])a_versions.ToArray(typeof(SPVersion));
> > > a_files.Add(a_fileNode);
> > > }
> > > if (a_files.Count > 0)
> > > a_returnNode.m_files = (SPTree [])a_files.ToArray(typeof(SPTree));
> > > }
> > > return a_returnNode;
> > > }
> > > }
> > > public class SPVersion
> > > {
> > > [XmlAttribute()] public string m_id;
> > > [XmlAttribute()] public string m_createdBy;
> > > [XmlAttribute()] public string m_createdOn;
> > > [XmlAttribute()] public string m_size;
> > > [XmlText()] public string m_comment;
> > > public SPVersion()
> > > {
> > > }
> > > }
> > > public class SPTree
> > > {
> > > [XmlAttribute()] public string m_url;
> > > [XmlAttribute()] public string m_checkedOutBy;
> > > [XmlTextAttribute()] public string m_displayName;
> > > public string m_currentVersion;
> > > public SPVersion [] m_versions;
> > > public SPTree [] m_folders;
> > > public SPTree [] m_files;
> > > public SPTree()
> > > {
> > > }
> > > }
> > >
> > >
> >
> >
>
>

Relevant Pages

  • Re: BizTalk Adapter for Sharepoint
    ... We use SP document lib for example to see what orders came in (we have nice ... I'll ask if this is a good practice in the sharepoint newsgroup... ... >>send docs to SP libraries, ... event handlers are fired only in document ...
    (microsoft.public.biztalk.general)
  • Re: Linking InfoPath to Sharepoint (SBS2003)
    ... It depends on your InfoPath form as to what will work. ... utilize InfoPath with Sharepoint via Form Libraries. ... The way the forms libraries work in Sharepoint is that they are effectively ...
    (microsoft.public.windows.server.sbs)
  • SharePoint memory leak
    ... TDC have a Sarbanes-Oxley site on SharePoint server, ... The w3wp.exe process uses an enormous amount of memory. ... But I looked at the virtual memory used by the w3wp worker process. ...
    (microsoft.public.sharepoint.portalserver.development)
  • Re: Public Folders and SharePoint 2003
    ... Goto the Virtual Server General Settings/E-Mail Enabled Document Libraries ... > enables to associate an Exchange Public Folder with a SharePoint Document ... This in effect, email enables SharePoint. ...
    (microsoft.public.sharepoint.portalserver)
  • Re: Will a large library cause WMP10 to slow system to a halt?
    ... 704 Megabytes Installed Memory ... of case likely would slow nearly everything down - not just the WMP. ... I have 2 250 Gb drives in a RAID 1 ... If you don't use libraries, how do you play music in WMP? ...
    (microsoft.public.windowsmedia.player)