Question

I’ve been trying to parse a wikipedia page in Python and have been quite successful using the API.

But, somehow the API documentation seems a bit too skeletal for me to get all the data.
As of now, I’m doing a requests.get() call to

http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=China&format=json&exintro=1

But, this only returns me the first paragraph. Not the entire page. I’ve tried to use allpages and search but to no avail. A better explanation of how to get the data from a wiki page would be of real help. All the data and not just the introduction as returned by the previous query.

Answer 1

You seem to be using the query action to get the content of the page. According to it’s api specs it returns only a part of the data. The proper action seems to be query.

Here is a sample

import urllib2
req = urllib2.urlopen("http://en.wikipedia.org/w/api.php?action=parse&page=China&format=json&prop=text")
content = req.read()
# content in json - use json or simplejson to get relevant sections.