Question

I have a web crawler built in C# (I know) and it has grown fairly sophisticated in handling many of the features normally handled by an actually web browser. That said, I have nothing that will parse the incoming HTML and process the embedded JavaScript commands on the page.

I have tried numerous approaches – from Noesis to Awesomium – but nothing appears to be working. I also made the mistake of using the WinForms embedded web browser control and the memory leaks under load (I am running Parallel Tasks) literally corrupted the CLR. That said, it was able to process the page as a normal browser and the resultant content was great – not viable, but the end result content was on point.

Is there nothing out there that will either take a target URL or, ideally, take in HTML content downloaded via an HttpWebRequest and process the embedded JavaScript commands?

Answer 1

1. http://www.mozilla.org/js/spidermonkey/
2. http://code.google.com/p/v8
3. KJS

Here is a list of JavaScript Engines. Also check ECMAScript engines.