Question

I am trying to grep some results pages for work, and then eventually print them out to an html website so someone does not have to manually look through each section.

How I would eventually use: I feed this function a result page, it greps through the 5 different sections, then I can do a html output (thats what that print substitute area is for) with all the different results.

OK MASSIVE EDIT I actually removed the old code because I was asking too many questions. I fixed my code taking some suggestions, but I am still interested in the advantage of using human-readable dict instead of just list. Here is my working code that gets all the right results into a ‘list of lists’, I then outputted the first section in my eventual html block

import urllib
import re
import string
import sys

def ipv6_results(input_page):
sections = ['/spec.p2/summary.html', '/nd.p2/summary.html',
            '/addr.p2/summary.html', '/pmtu.p2/summary.html', 
            '/icmp.p2/summary.html']
variables_output=[]                                                    
for s in sections:
    temp_list = []
    page = input_page + s
    #print page
    url_reference = urllib.urlopen(page)
    html_page = url_reference.read()
    m = re.search(r'TOTAL</B></TD><TD>:</TD><TD>([0-9,]+)', html_page)
    temp_list.append(int(m.group(1)) )
    m = re.search(r'PASS</B></TD><TD>:</TD><TD>([0-9,]+)', html_page)
    temp_list.append(int(m.group(1)))
    m = re.search(r'FAIL</FONT></B></TD><TD>:</TD><TD>([0-9,]+)', html_page)
    temp_list.append(int(m.group(1)))
    variables_output.append(temp_list)

#print variables to check them :)
print "------"
print variables_output


print "Ready Logo Phase 2"
print "Section                 | Total | Pass | Fail |"
#this next part is eventually going to output an html block
output =  string.Template("""
1 - RFC2460-IPv6 Specs $spec_total $spec_pass $spec_fail
""")
print output.substitute(spec_total=variables_output[0][0], spec_pass=variables_output[0][1],
                        spec_fail=variables_output[0][2])


return 1

imagine the tabbing is correct 🙁 I wish this was more like paste bin, suggestions welcome on pasting code in here

Answer 1

Generally, you don’t declare the shape of the list first, and then fill in the values. Instead, you build the list as you discover the values.

Your variables has a lot of structure. You’ve got inner lists of 3 elements, always in the order of ‘total’, ‘pass’, ‘fail’. Perhaps these 3-tuples should be made namedtuples. That way, you can access the three parts with humanly-recogizable names (data.total, data.pass, data.fail), instead of cryptic index numbers (data[0], data[1], data[2]).

Next, your 3-tuples differ by prefixes: 'spec', 'nd', 'addr', etc.
These sound like keys to a dict rather than elements of a list.

So perhaps consider making variables a dict. That way, you can access the particular 3-tuple you want with the humanly-recognizable variables['nd'] instead of variables[1]. And you can access the nd_fail value with variables['nd'].fail instead of variables[1][2]:

import collections
# define the namedtuple class Point (used below).
Point = collections.namedtuple('Point', 'total pass fail')

# Notice we declare `variables` empty at first; we'll fill in the values later.
variables={}
keys=('spec','nd','addr','pmtu','icmp')
for s in sections:
    for key in keys:
        page = input_page + s
        url_reference = urllib.urlopen(page)
        html_page = url_reference.read()
        m = re.search(r'TOTAL</B></TD><TD>:</TD><TD>([0-9,]+)', html_page)
        ntotal = int(m.group(1)) 
        m = re.search(r'PASS</B></TD><TD>:</TD><TD>([0-9,]+)', html_page)
        npass = int(m.group(1))
        m = re.search(r'FAIL</FONT></B></TD><TD>:</TD><TD>([0-9,]+)', html_page)
        nfail = int(m.group(1))

        # We create an instance of the namedtuple on the right-hand side
        # and store the value in `variables[key]`, thus building the 
        # variables dict incrementally.
        variables[key]=Point(ntotal,npass,nfail)