Question

So this is the script i am running which gives output fine in windows but in ubuntu,
it just prints a empty list

import urllib2
import os
import re
import csv
from bs4 import BeautifulSoup

useragent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1309.0 Safari/537.17'

def main():
    # lib-talkingpointsmemo.py
    archive = 'http://talkingpointsmemo.com/archive.php'    
    getweeklinks(archive)

def getweeklinks(archivelink):
    print 'something'
    urls = []
    request = urllib2.Request(archivelink, headers={'User-agent': useragent})
    webpage = urllib2.urlopen(request).read()   
    soup = BeautifulSoup(webpage)
    anchors = soup('a') 
    print anchors
    for a in anchors:
        print a['href']

if __name__ == '__main__' : main()

and the output :

something
[]

Whats wrong? I am using Ubuntu 12.04.1 LTS

Answer 1

hmmm… nothing is wrong with your script, it is working fine for me on Ubuntu, except for I’m using: Ubuntu 10.04.2 LTS, and Python 2.6.5

shot in the dark, but maybe try…

soup = BeautifulSoup(webpage,"html.parser")

…to make sure your using the same parser between your windows and ubuntu tests. You may also want to try some of the other parser options