Re: downloading a single file using multiple threads

On Wed, 28 Mar 2007 08:51:09 -0700, Willy Denoyette [MVP] <willy.denoyette@xxxxxxxxxx> wrote:

A *single file* download from an FTP or HTTP server? How do you indicate what chunk of the file you want when say using FTP? As far as I know this is not part of the FTP neither of HTTP protocol.

See, for example, the "Range" field in HTTP.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1

(I'm guessing that the above link may actually answer the question for the OP)

Per file? I mean, retrieve a *single file* from multiple servers in chunks? Say the first 10 MB's from server A the next 2MB from server B, the next 3 MB from server C? Never heard of something like this.

Well, it does happen. Most larger servers monitor the client IP address and refuse additional connections, and with smaller servers doing this sort of thing is considered anti-social, since it intentionally bypasses per-client throttling that has been set up. But it's true that there are "download manager" programs that do exactly what Jon is referring to.

All I know that there are download managers that download file A from server A, file B from server B etc.. in case of multi-file downloads.

Note that there's nothing to stop a download manager from retrieving different parts of the same file from multiple servers as well. Assuming an identical file stored on various mirrors, it doesn't matter which mirror a given part of the file comes from.

Pete
.

Relevant Pages

  • Re: Partitioned decompression
    ... | This big compressed file will be transfered between a server and the ... | this chunk is requested by other peer, ... name, creation date, and comments to the client from the server. ... the server runs a checksum for every 100 KB ...
    (comp.compression)
  • Re: LWP get but only get the first 25% then exit/stop_retrieveing
    ... read, at least, the first chunk, entirely, regardless of length. ... coming out of a server, but not an Apache server as prior discussed. ... Use of exit will break the connection, for your script, upon reading the first ... Without an exit, the read process will continue until the final chunk arrives. ...
    (perl.beginners)
  • Re: missing charset or bung server settings?
    ... >example server outputs below, and help me understand what is wrong. ... that are causing the XML to be invalid. ... using the output directly, your client is broken, as it's not decoding the ... Chunk size here. ...
    (comp.lang.php)
  • "Cannot access a closed Stream" when using DimeAttachment
    ... public class MyDimeService: System.Web.Services.WebService ... (from client to server). ... We can, ofcource, for every chunk, create a new stream and write data ...
    (microsoft.public.dotnet.framework.aspnet.webservices)
  • Re: Help with OutputStream.send() method. I want to know how much data this method already sent.
    ... First I have to get the sockWriter: ... progress bar showing how much data has already been sent to the server. ... don't send all of the bytes in one chunk. ...
    (comp.lang.java.programmer)