Question

I have the following given html structure

<li class="g">
 <div class="vsc">    
  <div class="alpha"></div>
  <div class="beta"></div>
  <h3 class="r">
   <a href="http://www.stackoverflow.com"></a>
  </h3>
 </div>
</li> 

The above html structure keeps repeating, what can be the easiest way to parse all the links(stackoverflow.com) from the above html structure using BeautifulSoup and Python?

Answer 1

BeautifulSoup 4 offers a convenient way of accomplishing this, using CSS selectors:

from bs4 import BeautifulSoup
soup = BeautifulSoup(html)
print [a["href"] for a in soup.select('h3.r a')]

This also has the advantage of constraining the selection by context: it selects only those anchor nodes that are children of a h3 node with class r.

Omitting the constraint or choosing one most suitable for the need is easy by just tweaking the selector; see the CSS selector docs for that.