Question

I am trying to get all the urls from an html page.
I have succeeded in getting the urls from the page itself but there are also javascripts that contain urls.
How do I get the urls from them?
I have been searching for a way for a while and i will appreciate your help.

Answer 1

If the URLs are just strings in the JavaScript code then you could extract them by matching everything that looks like a URL in the “script” tag text. E.g.:

List<URL> urls = new ArrayList<URL>();
Pattern p = Pattern.compile(myUrlPattern);
Matcher m = p.matcher(eachScriptTagText);
while (m.find()) {
  urls.add(m.group());
}

The regular expression for a URL can be found easily on the internet.