Home/ Questions/Q 6224341
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-24T08:39:13+00:00

I am pulling JSON data from several remote servers concurrently over HTTP, using a

  • 0

I am pulling JSON data from several remote servers concurrently over HTTP, using a WCF service on both the client and server endpoints. I’m noticing that for each successive request that starts asynchronously the length of time that http request takes is generally increasing, even if the amount of data is not necessarily increasing. In other words if I start 12 thread pool threads (using Func<>.BeginInvoke) then each request, after being timed, is showing up in my logs as such:

  :HttpRequest invoked. Elapsed: 325ms
  :HttpRequest invoked. Elapsed: 27437ms
  :HttpRequest invoked. Elapsed: 28642ms
  :HttpRequest invoked. Elapsed: 28496ms
  :HttpRequest invoked. Elapsed: 32544ms
  :HttpRequest invoked. Elapsed: 38073ms
  :HttpRequest invoked. Elapsed: 41231ms
  :HttpRequest invoked. Elapsed: 47914ms
  :HttpRequest invoked. Elapsed: 45570ms
  :HttpRequest invoked. Elapsed: 61602ms
  :HttpRequest invoked. Elapsed: 53567ms
  :HttpRequest invoked. Elapsed: 79081ms

The process is pretty simple. I am simply starting each request in a loop and then calling .WaitAll() on all of the operations before using the consolidated data.

It looks like the Http requests are taking way longer than they should even with small amounts of data. In fact the difference between small and large amounts of data appears minimal overall. Would this sort of bottleneck be due to concurrent http requests having to share bandwidth, or is there a threading / context-switching issue possible here? Just looking to be pointed in the right direction.

EDIT — Just for clarity, I ran the same process synchronously and here are the results:

  :HttpRequest invoked. Elapsed: 20627ms
  :HttpRequest invoked. Elapsed: 16288ms
  :HttpRequest invoked. Elapsed: 2273ms
  :HttpRequest invoked. Elapsed: 4578ms
  :HttpRequest invoked. Elapsed: 1920ms
  :HttpRequest invoked. Elapsed: 564ms
  :HttpRequest invoked. Elapsed: 1210ms
  :HttpRequest invoked. Elapsed: 274ms
  :HttpRequest invoked. Elapsed: 145ms
  :HttpRequest invoked. Elapsed: 21447ms
  :HttpRequest invoked. Elapsed: 27001ms
  :HttpRequest invoked. Elapsed: 1957ms

The total time (because its synchronous) went up, however you can see clearly that each individual request is generally faster. Unfortunately I dont know of any way to isolate the problem — but my guess is that its a bandwidth sharing issue between the threads.

So I some more straightforward question I have is:

1) If I use a non-threadpool thread, would this improve

2) Should I group the operations into only a few threads, rather than each request having its own?

3) Is this a standard problem when trying to concurrently download data over Http?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    The best answer to this question I ultimately came up with came from this link:

    http://blogs.msdn.com/b/wenlong/archive/2010/02/11/why-are-wcf-responses-slow-and-setminthreads-does-not-work.aspx

    Which applies if you are calling the same servive method repeatedly.

    By tweaking ThreadPool.SetMinThreads() on both client and server I was able to get improved results.

    However there are still alot of slowness issues with WCF and transferring over HTTP that I haven’t really gotten to the bottom of.

    • 0