Home/ Questions/Q 8291007
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-08T12:59:09+00:00

Whats the most pythonic way to make Mechanize follow links ( br.follow_link ) that

  • 0

Whats the most pythonic way to make Mechanize follow links (br.follow_link) that are located inside a certain div? I know how to do it with a bit of help from BeautifulSoup but is there a way of doing it with Mechanize?

Sample div:

<div id="blah_links">
 <a href="LINK1" class="active">1</a> |
 <a href="LINK2">2</a> |
 <a href="LINK3">3</a> |
 <a href="LINK4">NEXT</a>
</div>
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I ran into similar problem recently and here is what i did

    url = "www.somewhere.com"
br = mechanize.Browser()
br.open(url)

encoded_data = UnicodeDammit(br.response().read(),isHTML=True).unicode
parser = lxml_html.fromstring(encoded_data)

soup_xpath = "//div[@id='BODYCON']//a/@href"
valid_links = soup.xpath(soup_xpath)
links  = [ link for link if link.url in valid_links ]
    • 0