Question

I use http://www.catonmat.net/blog/python-library-for-google-search/ to make
queries at Google search engine but the number of results I get is limited to
10. I used the results_per_page property and I set it to 50, 100 etc. but the
number of results didn’t changed. Is there a way to get more results?
Is there another python lib without these constraints for Google searching?

Thank you in advance,
Thanasis

Answer 1

You will need to loop through the pages: listed below is an example; remember to place a thread sleep between each call to Google: otherwise your IP will be blocked if your search is large (getting it unblocked is a real hassle and your ISP may get on your case).

from xgoogle.search import GoogleSearch, SearchError
try:
  page = 1
  gs = GoogleSearch("foo bar")  
  gs.results_per_page = 100
  results = []
  while page < 10:
      gs.page = page
      results += gs.get_results()
      page += 1
except SearchError, e:
  print "Search failed: %s" % e
for res in results:
    print res.url