Question

Some time ago I created an application to dynamically capture data from an asp site navigating it, parsing the html pages I got and storing the selected data into a database.

Now I need to do the same again but this time the web site is developed using AJAX and I don’t know how to face the problem. Any ideas are welcome.

Thanks.

Answer 1

I’d suggest two ways of solving you problem:

1) if you create a crawler for one particular site with tons of data to retrieve, write these requests manually (using WebRequest class, for example)

2) if you need universal solution, try some GUI testing tools (Selenium, Telerik WebAii etc.) to run browser into site, so JS and AJAX will be executed by browser.

Depends on you.