Question

So lets say I was trying to get the link to a certain image, like this:

from bs4 import BeautfiulSoup
import urlparse

soup = BeautifulSoup("http://examplesite.com")
for image in soup.findAll("img"):
    srcd = urlparse.urlparse(src)
    path = srcd.path # gets the path
    fn = os.path.basename(path) # gets filename

# lets say the webpage i was scraping had their images like this:
# <img src="../..someimage.jpg" />

Is there any easy way to get the full url from that? Or will I have to use regex?

Answer 1

Use urlparse.urljoin:

>>> import urlparse
>>> base_url = "http://example.com/foo/"
>>> urlparse.urljoin(base_url, "../bar")
'http://example.com/bar'
>>> urlparse.urljoin(base_url, "/baz")
'http://example.com/baz'