We have an in-house application that pulls some data from some of Amazon’s pages occassionally (We know they have APIs for certain operations… what we’re doing requires some custom info not included in the APIs). We have never had a problem pulling their pages, but suddenly Amazon is returning “(503) Server Unavailable” on pretty much every request, and this has been happening for several days, so we doubt it’s a temporary thing. Even something as simple as this:
System.Net.WebClient client = new System.Net.WebClient();
string data = client.DownloadString(new Uri("http://www.amazon.com/Bose-Companion-multimedia-speaker-Graphite/dp/B000HZBR64/"));
The strange thing is that these pages load just fine in a web browser, but any time we try to pull them through code, it is failing.
What could cause these functions to fail? Is it possible that they changed something on their end and that we need to do some custom logic with our calls?
After some further testing, it turns out that this was happening because Amazon needs the Accept parameter of the HttpWebRequest to be specifically set. When setting it to:
Everything worked fine. This is a recent change, so they must have altered something on their end.