Question

I am trying to write a function that will allow me to browse for a CSV file, and then parse that CSV file, before sending off each value to the Google Search API (that bit is already written).

So right now I’ve got this:

def loadtemplate():
    filename = tkFileDialog.askopenfilename(filetypes = (("CSV files", "*.csv")
                                                             ,("Text Files", "*.txt")
                                                             ,("All files", "*.*") ))
    if filename: 
            try: 
                csvfile = csv.reader(open(filename, 'rb'), delimiter=',')
                for row in csvfile:
                    for x in row:
                            generate(x)
            except: 
                tkMessageBox.showerror("Open Source File", "Failed to read file \n'%s'"%filename)
                return

And my CSV file looks like this:

seo,company name,drupal,wordpress themes,awesome web design

Nothing too crazy. Anyway, I’m getting this error:

Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 1410, in __call__
    return self.func(*args)
  File "C:/Python27/Projects/Google Searcher/realsearcher.py", line 20, in loadtemplate
    generate(x)
  File "C:/Python27/Projects/Google Searcher/realsearcher.py", line 31, in generate
    gs = gs['cursor']
TypeError: 'NoneType' object is not subscriptable

It seems like somehow a value is being set to None? But I keep either trying to use a conditional where if x == None:, it’ll not allow the query to go through or try to change the CSV file to where nothing like that should be parsed.

What’s going on here, and how do I fix it?

PS – this is what the variable row looks like:

['seo', 'company name', 'drupal', 'wordpress themes', 'awesome web design']

Here is generate() (I used duplicate code because I felt that writing something that would do both solutions would take longer and was unnecessary because this project won’t be expanded):

def generate(item):
    infoget = urllib.quote(item)
    infoquote = '"' + infoget + '"' 
    response = urllib2.urlopen("http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=" + infoget)
    gs = simplejson.load(response)
    gs = gs['responseData']
    gs = gs['cursor']
    gs = gs['estimatedResultCount']
    print gs
    response = urllib2.urlopen("http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=" + infoquote)
    gs = simplejson.load(response)
    gs = gs['responseData']
    gs = gs['cursor']
    gs = gs['estimatedResultCount']
    print gs

Answer 1

It should be obvious from the traceback that the problem has nothing whatsoever to do with the CSV loading, but comes from the generate function. We can see from the code for that function that you’re querying the Google Search API (for every item in every line!). However, in various cases, you’re getting a blank result back.

You should rewrite the generate function to be a little more sensible, without spamming Google (they don’t like it), and dealing with failures a sensibly – checking that the nested values exist in the JSON before relying on them.