Home/ Questions/Q 205607
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-11T17:34:17+00:00

I am writing a C++ application and would like to request several data files

  • 0

I am writing a C++ application and would like to request several data files through a HTTP GET request simultaneously, where should I look to get started (needs to be cross-platform).

  1. Run Application
  2. Create a list of URLs { “http://host/file1.txt“, “http://host/file2.txt“, “http://host/file3.txt“}
  3. Request all the URLs simultaneously and load the contents to variables (don’t want disk writes). Each file has about 10kB of data.

What libraries would you recommend I use? libcurl? curlpp? boost asio? would I need to roll-my-own multi threading to request all the files simultaneously? is there a easier way?

Edit: I will need to request about 1000 GET requests simultaneously. Most likely I will do this in batches (100 at a time, and creating more connections as made one are completed).

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I would recommend libcurl. I’m not super-familiar with it, but it does have a multi-interface for performing multiple simultaneous HTTP operations.

    Depending on what solution you go with, it’s possible to do asynchronous I/O without using multithreading. The key is to use the select(2) system call. select() takes a set of file descriptors and tells you if any of them have data available. If they do, you can then proceed to use read(2) or recv(2) on them without worrying about blocking.

    • 0