Home/ Questions/Q 9004215
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-16T00:51:11+00:00

I am working on a quick tool that grabs images off of a web

  • 0

I am working on a quick tool that grabs images off of a web page. Currently, I use a WebClient to get the source data of the page, parse the image URLs out of the source, and download them individually to a folder using WebClient.DownloadFile. This can take quite a while.
I understand that most of the time needed is due to my connection and downloading the data.
Are there any other, more efficient ways of going about this, whether it be a C# HTML parsing library or something else?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You can use multiple threads, which will open multiple concurrent HTTP connections to the web page.

    One good approach would be to implement a Producer/Consumer pattern: have one thread that gets and parses the HTML containing the images, then queues the image URLs into something like a BlockingCollection. Have multiple threads read the image URLs from the queue and download the images concurrently.

    http://msdn.microsoft.com/en-us/library/dd997371.aspx

    If you’re up for cutting edge, this class if problem is ideally suited for TPL Dataflow (alternative to a BlockingCollection).

    http://msdn.microsoft.com/en-us/devlabs/gg585582.aspx

    • 0