Question

I have a HTML page that displays a few values. I also have a little app that displays data from some other pages I have, but these other pages are JSON, not HTML.
I want to consume these values from the HTML page, convert to JSON, then output.

The reason I want to do this is so that I can simply reuse my code, and just change the URL, or even dynamically create it.

I made the HTML page as plain as possible, so as to strip out all the junk in order to make the regex more basic.

Here is the HTML:

<div class="BlockA">
    <h4>BlockA</h4>
    <div class="name">John Smith</div>
    <div class="number">2</div>
    <div class="name">Paul Peterson</div>
    <div class="number">14</div>
</div>

<div class="BlockB">
    <h4>BlockB</h4>
    <div class="name">Steve Jones</div>
    <div class="number">5</div>
</div>

Both blocks will have varying numbers of elements,depending on a few factors.

Here is my python:

def index(request, toGet="xyz"):
    file = urllib2.urlopen("http://www.mysite.com/mypage?data="+toGet)
    data = file.read()
    dom = parseString(data)
    rows = dom.getElementsByTagName("BlockA")[0]
    readIn = ""
    for row in rows:
        readIn = readIn+json.dumps(
            {'name': row.getAttribute("location"),
            'number': row.getAttribute("number")},
            sort_keys=True,
            indent=4)+","
    response_generator = ( "["+readIn[:-1]+"]" )
    return HttpResponse(response_generator)

So this is basically reading the values (actually, the source is XML in this case), looping through them, and outputting all the values.

If someone can point me in the right direction, it would be much appreciated. For example, reading in the tags like “BlockA” and then the tags “name” and “number”.

Thanks.

Answer 1

If you truly need to parse an HTML page in Python, you should be using Beautiful Soup. I question whether you really should be doing this though. Are the HTML pages and JSON outputs using the same Django instance? Are they all apart of the same project?

If they are apart of the same project, then you can use something like django-piston which is a RESTful framework for python. This will allow you to define the data that should be exposed, and output in multiple formats such as HTML/Django Template, JSON, XML, or YAML. You can also create your own emitters to output as a different format.

That way, you can expose a particular URL as a regular template, or get the same data as JSON would will be much easier to parse than HTML.

Sorry if I’m misunderstanding your problem. But it really does sound like you want to expose a view as several different formats, and a RESTful framework will help with that.