Re: downloading a single file using multiple threads

Willy Denoyette [MVP] <willy.denoyette@xxxxxxxxxx> wrote:
See, for example, the "Range" field in HTTP.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1

I know that HTTP 1.1 supports range requests, but this is not exactly my point.
The multi part requests in HTTP1.1 are meant to request (for very specic application
purposes) a single part or multiple parts in a single request, but you can't (AFAIK)
requests multiple parts in parallel from multiple client threads.

Why not? They'd be multiple requests, each requesting a single part to
form a whole. What's to stop it working?

Note that there's nothing to stop a download manager from
retrieving different parts of the same file from multiple servers
as well. Assuming an identical file stored on various mirrors, it
doesn't matter which mirror a given part of the file comes from.

That's true, but these will use dedicated protocols don't they? The
clients also should have multiple NIC's installed connected over
segmented LAN's and/or routers to take some speed advantage of the
parallelism.

Why shouldn't it work over multiple HTTP servers (using range headers)?
It depends where the network bottleneck is, of course, but if I can get
from the UK to (say) the US with 1Mbps and to Africa with 1Mbps, in
parallel, it makes sense to fetch at an overall rate of 2Mbps rather
than 1...

--
Jon Skeet - <skeet@xxxxxxxxx>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
.

Relevant Pages

  • Re: Using Cobol when writing CGIs for the OSU web server.
    ... OSU web server. ... Why do you have that OSU ... This makes requests such as /scr/chocolate.scp to the OAJFOSU decnet ... multiple connections at the same time with threads or ASTs. ...
    (comp.os.vms)
  • Re: Confused about ASP, "sessions", and queuing of multiple requests
    ... cookies (include ASP session state cookie) with the first IE instance. ... and generating multiple HTTP requests from a single IE ... :> running IIS on a multiprocessor server. ...
    (microsoft.public.inetserver.asp.general)
  • Re: Confused about ASP, "sessions", and queuing of multiple requests
    ... cookies (include ASP session state cookie) with the first IE instance. ... and generating multiple HTTP requests from a single IE ... :> running IIS on a multiprocessor server. ...
    (microsoft.public.inetserver.iis)
  • Re: downloading a single file using multiple threads
    ... The multi part requests in HTTP1.1 are meant to request a single part or multiple parts in a single request, but you can't requests multiple parts in parallel from multiple client threads. ... I know that down-load managers claim to work over HTTP, but that doesn't mean they support multi-part parallel request handling over the same or multiple connections, I don't even know if the protocol allows you to issue new range request when you have a range requests pending. ... If on one connection the client asks for the first megabyte and on a second connection the same client asks for the second megabyte, then if the server is capable of servicing both requests at the same time, there's no reason the client can't wind up receiving both the first and second megabytes in parallel. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Confused about ASP, "sessions", and queuing of multiple requests
    ... ASP is exactly what we have, with multiple of these IMGs on one page. ... these multiple requests in a "queuing" fashion vs. not in a queuing fashion. ... running IIS on a multiprocessor server. ... But on our production server, instead, for a given user operation at the ...
    (microsoft.public.inetserver.iis)
Loading