Re: downloading a single file using multiple threads

"Peter Duniho" <NpOeStPeAdM@xxxxxxxxxxxxxxxx> wrote in message news:op.tpwp6khg8jd0ej@xxxxxxxxxxxxxxxxxxxxxxx
On Wed, 28 Mar 2007 08:51:09 -0700, Willy Denoyette [MVP] <willy.denoyette@xxxxxxxxxx> wrote:

A *single file* download from an FTP or HTTP server? How do you indicate what chunk of the file you want when say using FTP? As far as I know this is not part of the FTP neither of HTTP protocol.

See, for example, the "Range" field in HTTP.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1


I know that HTTP 1.1 supports range requests, but this is not exactly my point.
The multi part requests in HTTP1.1 are meant to request (for very specic application purposes) a single part or multiple parts in a single request, but you can't (AFAIK) requests multiple parts in parallel from multiple client threads.

(I'm guessing that the above link may actually answer the question for the OP)

Per file? I mean, retrieve a *single file* from multiple servers in chunks? Say the first 10 MB's from server A the next 2MB from server B, the next 3 MB from server C? Never heard of something like this.

Well, it does happen. Most larger servers monitor the client IP address and refuse additional connections, and with smaller servers doing this sort of thing is considered anti-social, since it intentionally bypasses per-client throttling that has been set up. But it's true that there are "download manager" programs that do exactly what Jon is referring to.

All I know that there are download managers that download file A from server A, file B from server B etc.. in case of multi-file downloads.

Note that there's nothing to stop a download manager from retrieving different parts of the same file from multiple servers as well. Assuming an identical file stored on various mirrors, it doesn't matter which mirror a given part of the file comes from.

That's true, but these will use dedicated protocols don't they? The clients also should have multiple NIC's installed connected over segmented LAN's and/or routers to take some speed advantage of the parallelism.

Willy.

.

Relevant Pages

  • Re: downloading a single file using multiple threads
    ... The multi part requests in HTTP1.1 are meant to request a single part or multiple parts in a single request, but you can't requests multiple parts in parallel from multiple client threads. ... I know that down-load managers claim to work over HTTP, but that doesn't mean they support multi-part parallel request handling over the same or multiple connections, I don't even know if the protocol allows you to issue new range request when you have a range requests pending. ... If on one connection the client asks for the first megabyte and on a second connection the same client asks for the second megabyte, then if the server is capable of servicing both requests at the same time, there's no reason the client can't wind up receiving both the first and second megabytes in parallel. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Confused about ASP, "sessions", and queuing of multiple requests
    ... cookies (include ASP session state cookie) with the first IE instance. ... and generating multiple HTTP requests from a single IE ... :> running IIS on a multiprocessor server. ...
    (microsoft.public.inetserver.asp.general)
  • Re: Confused about ASP, "sessions", and queuing of multiple requests
    ... cookies (include ASP session state cookie) with the first IE instance. ... and generating multiple HTTP requests from a single IE ... :> running IIS on a multiprocessor server. ...
    (microsoft.public.inetserver.iis)
  • Re: problem in smtp server
    ... Thank you very much, now my ftp is working, and even the smtp seems to ... > run an ident server, which I'd be pretty certain you don't!) ... > working becuase you're serving active FTP requests (rather than ...
    (comp.unix.bsd.freebsd.misc)
  • Re: Confused about ASP, "sessions", and queuing of multiple requests
    ... ASP is exactly what we have, with multiple of these IMGs on one page. ... these multiple requests in a "queuing" fashion vs. not in a queuing fashion. ... running IIS on a multiprocessor server. ... But on our production server, instead, for a given user operation at the ...
    (microsoft.public.inetserver.iis)