Re: Iterating and downloading a URL that lists files?

Tech Tip: Click here to run a free scan for Windows Errors and optimize PC performance

Hi,

Alexey Smirnov wrote:
On Aug 2, 5:15 am, "Jim Bancroft" <sd...@xxxxxxx> wrote:
Is there a way to open a URL, one that shows nothing other than a directory
listing, and iteratively go through the list to download the files?

Hi Jim,

you can use a HttpWebRequest with a StreamReader to get the page from
the URL, then the content of the page can be parsed with a regular
expression to get the list of all links and then again use
HttpWebRequest to get the files...

Hope it helps

I did that a long time ago with a Java applet. I then discovered that the page format was different in Netscape (back then) and IE. I hope things are more standard now!

HTH,
Laurent
--
Laurent Bugnion [MVP ASP.NET]
Software engineering, Blog: http://www.galasoft.ch
PhotoAlbum: http://www.galasoft.ch/pictures
Support children in Calcutta: http://www.calcutta-espoir.ch
.

Relevant Pages

  • Re: Asynchronous HTTP Download
    ... I just wrote one for a client using the HttpWebRequest class. ... I created a download worker thread that requested a chunk of data from the ... > asynchronous using a timer calling the Read method of the StreamReader ...
    (microsoft.public.dotnet.languages.vb)
  • Re: Iterating and downloading a URL that lists files?
    ... listing, and iteratively go through the list to download the files? ... you can use a HttpWebRequest with a StreamReader to get the page from ... HttpWebRequest to get the files... ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: Timeout bei Download einer Datei - kein Grund ersichtlich
    ... Nach dem Download muss der Stream explizit mit "st.close" geschlossen werden. ... req = (HttpWebRequest) WebRequest.Create; ... Da ich den Download je in einem separaten Thread ausführe (bzw. in 1-5 Threads, je nach Anzahl der Bilder), vermute ich, dass DownloadDataAsync auch keine Besserung bringt. ... Dabei tritt die selbe Exception auf. ...
    (microsoft.public.de.german.entwickler.dotnet.csharp)
  • Re: Timeout bei Download einer Datei - kein Grund ersichtlich
    ... req = (HttpWebRequest) WebRequest.Create; ... Da ich den Download je in einem separaten Thread ausführe (bzw. in 1-5 ... Threads, je nach Anzahl der Bilder), vermute ich, dass ... Dabei tritt die selbe Exception auf. ...
    (microsoft.public.de.german.entwickler.dotnet.csharp)
  • Re: Access Denied to assembly.
    ... > HttpWebRequest request = ... > StreamReader responseData = new StreamReader( ... Policy not being applied to reference at this time (private, custom, ... > partial, or location-based assembly bind). ...
    (microsoft.public.dotnet.framework)