Question

I’m working on a Python script that transforms this:

foo
bar

Into this:

[[Component foo]]
[[bar]]

The script checks (per input line) if the page “Component foo” exists. If it exists then a link to that page is created, if it doesn’t exist then a direct link is created.

The problem is that I need a quick & cheap way to check if a lot of wiki pages exist.I don’t want to (try to) download all the ‘Component’ pages.

I already figured out a fast way to do this by hand: Edit a new wiki page. paste all the ‘component’ links into the page, press preview, and then save the resulting preview HTML page. The resulting HTML file contains a different link for existing pages than for non-existing pages.

So to rephrase my question: How can I save a mediawiki preview page in Python?

(I don’t have local access to the database.)

Answer 1

You can definitely use the API to check if a page exists:

# assuming words is a list of words you wish to query for
import urllib

# replace en.wikipedia.org with the address of the wiki you want to access
query = "http://en.wikipedia.org/w/api.php?action=query&titles=%s&format=xml" % "|".join(words)
pages = urllib.urlopen(query)

Now pages you will contain xml like this:

<?xml version="1.0"?><api><query><pages>

   <page ns="0" title="DOESNOTEXIST" missing="" />

   <page pageid="600799" ns="0" title="FOO" />

   <page pageid="11178" ns="0" title="Foobar" />

</pages></query></api>

Pages which don’t exist will appear here but they have the missing=”” attribute set, as can be seen above. You can also check for the invalid attribute to be on the save side.

Now you can use your favorite xml parser to check for these attributes and react accordingly.

See also: http://www.mediawiki.org/wiki/API:Query