I would like to use the Mechanize python library to follow certain links in a website, but the only links I’m interested in are the ones in a <div> tag. This question is related, but they achieve it using the lxml parser, which I’m not familiar with, I’m more comfortable using BeautifulSoup.
I’ve located the relevant links using BeautifulSoup already, but I don’t know how to use Mechanize (or something else) to follow these links. Is there a way to pass a string to Mechanize so that it will follow it?
the simple
open()should be enough: