Re: WriteFile buffers in memory

From: Tim Greenfield (timg_nospam_at_paloalto.com)
Date: 10/04/04 

Date: Mon, 4 Oct 2004 16:43:21 -0700

Thanks for the threading info. A max of 100 threads per CPU still seems
really low... 101 users all hitting a web page that takes a long time but
uses very few resources doesn't seem all that out of the ordinary. Oh well.

How do you suppose Response.WriteFile returns immediately? If I could get my
own code to behave like .WriteFile (but not tying up memory) then I could
have the best of both worlds. Maybe?

"bruce barker" <nospam_brubar@safeco.com> wrote in message
news:e2dfpYmqEHA.348@TK2MSFTNGP15.phx.gbl...
> you are stuck tying up a couple threads per request. you can bump up the
> i/o
> and workpool thread count. asp.net sets a max i/o thread count to 100 per
> cpu.
>
> what you are dealing with
>
> 1) iis keeps a thread pool to manage i/o connections to clients
> 2) iis run an isapi filter that handles asp.net pages
> 3) the asp.net filter keeps a pool of thread to manage i/o between it and
> asp.net. the filter uses named pipes to talk to the asp.net worker
> process.
> 4) the asp.net worker process keeps a pool of threads to talk to the
> asp.net
> filter over the named pipe connection
> 5) the asp.net worker process keeps a pool of threads to actually process
> a
> request.
>
> so with a file download, you will tieup:
>
> 1) an asp.net thread (until the last i/o flush)
> 2) an asp.net i/o thread (until flushed to the filter)
> 3) an asp.net i/o filter thread (until flushed to iis)
> 4) an iis i/o thread. (until i/o keep alive is canceled/times out)
>
> if you need high performance, you should look at writing your own isapi
> filter.
>
> -- bruce (sqlwork.com)
>
>
> "Tim Greenfield" <timg_nospam@paloalto.com> wrote in message
> news:u7oM47lqEHA.2764@TK2MSFTNGP11.phx.gbl...
>> Hi, I have a problem I'm hoping is not too unusual. I'm trying to push a
>> large file (50MB) out to the client from an ASP page. The reason I'm
>> using
>> an ASP page is so I can start the reading at a particular offset. This
>> ASP
>> page needs to be scaleable as it needs to support thousands of requests
> per
>> hour. I've found 2 solutions so far but both have their problems:
>> 1) Using Response.WriteFile(filename, offset, length). This seems like
>> the
>> perfect solution except that it puts the entire contents of the file in
>> memory per request... ouch! Anyone know of a way to put the file in some
>> global cache so it don't fill up my RAM per session? I could afford
> filling
>> up RAM per file as there are only a handful of files that will be
>> downloaded.
>> 2) Writing the file 1K at a time to the output buffer w/ buffering
> disabled.
>> This works great; each request barely takes up any memory or CPU.
>> However,
>> it seems to tie up 1 precious ASP thread per request. And it turns out
> that
>> ASP threads are very precious... they only give you 25 per processor by
>> default. Within a minute I'm out of threads and users get a "Server To
> Busy"
>> error. I can work around this by adding:
>> <httpRuntime appRequestQueueLimit="50000" /> to web.config
>> But all it does is make the 26th user wait for someone to finish their
>> download before granting access instead of giving them an error. Anyone
> know
>> how to increase the size and if so, am I treading on thin ice by
> increasing
>> it to some huge number like 1000?
>>
>> Another solution I've been pursuing is to try to take the best of both
>> worlds. If I could create a module that uses the 1K at a time technique
>> to
>> avoid memory hogging but get it return immediately by using
> multi-threading,
>> I think I could solve my problem. Unfortunately, I haven't been able to
> get
>> it to work yet.
>> ' from my .aspx file:
>> Dim SendFile as New SendFile
>> SendFile.Response = Response
>> SendFile.Filename = Filename
>> SendFile.Offset = Offset
>> Dim t as New Threading.Thread(AddressOf SendFile.Go)
>> t.Start()
>>
>> Can anyone offer any wisdom? I can't imagine I'm the first one to need to
> do
>> this. Thanks!
>>
>> -- Tim
>>
>>
>
>

Relevant Pages

  • Re: WriteFile buffers in memory
    ... you are stuck tying up a couple threads per request. ... asp.net sets a max i/o thread count to 100 per ... iis run an isapi filter that handles asp.net pages ... > an ASP page is so I can start the reading at a particular offset. ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: IIS Extensions in URL causes filter to break.
    ... If your filter operates the way you say, ... IIS can only say that one or more of the events contain ... every request, and maybe more than once, depending on whether the URL causes ... As for the order of handling between ISAPI Extensions and Filters - Filters ...
    (microsoft.public.inetserver.iis.security)
  • RE: iis crash/hang agent
    ... ;;; which is represented by a request that takes too long to complete. ... In the case of a hang, the agent can write a log file ... Configure the IISCHAgent.dll as a global filter in IIS by running ... Specifies the maximum number of CrashAgent logs to keep. ...
    (microsoft.public.inetserver.iis)
  • Re: Bybass HTTP ( extension files ) in ISA 2004
    ... The request was rejected by the HTTP filter. ... Contact your ISA Server administrator. ...
    (Bugtraq)
  • Re: PKZIPC, ASP and WSH
    ... does NOT wait for it to finish processing before completing the ASP page ... Go to Services control panel applet and change both the "IIS Admin ... make a request to your ASP page ... Dynamic File, Directly Executable -- these are CGI EXE and ISAPI DLLs ...
    (microsoft.public.scripting.wsh)
Loading