Question

I want know which programming language provides good number of libraries to program a web bot?
Something like crawling a web page for data. Say I want fetch weather for weather.yahoo.com website.

Also will the answer be same for a AI desktop bot?

Answer 1

Here is how you could do it in Python:

from urllib2 import urlopen
from BeautifulSoup import BeautifulSoup
soup=BeautifulSoup(urlopen("http://weather.yahoo.com/").read())
for x in soup.find(attrs={"id":"myLocContainer"}).findAll("li"):
  print x.a["title"], x.em.contents

Prints:

Full forecast for Chicago, Illinois, United States (Haze) [u’35…47 °F’]
Full forecast for London, Greater London, England (Light Rain) [u’43…45 °F’]
Full forecast for New York, New York, United States (Partly Cloudy) [u’42…62 °F’]
Full forecast for San Francisco, California, United States (Partly Cloudy) [u’51…70 °F’]