Question

This question is a supplement to an earlier question. If you need further background, you can check out the original question here:

Populating Python list using data obtained from lxml xpath command.

I have incorporated @ihor-kaharlichenko ‘s excellent suggestion (from my original question) into modified code, here:

from lxml import etree as ET
from datetime import datetime

xmlDoc = ET.parse('http://192.168.1.198/Bench_read_scalar.xml')

response = xmlDoc.getroot()
tags = (
'address',
'status',
'flow',
'dp',
'inPressure',
'actVal',
'temp',
'valveOnPercent',
)

dmtVal = []

for dmt in response.iter('dmt'):
    val = [str(dmt.xpath('./%s/text()' % tag)) for tag in tags]
    val.insert(0, str(datetime.now())) #Add timestamp at beginning of each record
    dmtVal.append(val)

for item in dmtVal:
    str(item).strip('[')
    str(item).strip(']')
    str(item).strip('"')

This last block is where I am having problems. The data I am getting for dmtVal looks like:

[['2012-08-16 12:38:45.152222', "['0x46']", "['0x32']", "['1.234']", "['5.678']", "['9.123']", "['4.567']", "['0x98']", "['0x97']"], ['2012-08-16 12:38:45.152519', "['0x47']", "['0x33']", "['8.901']", "['2.345']", "['6.789']", "['0.123']", "['0x96']", "['0x95']"]]

However, I really want the data to look like this:

[['2012-08-16 12:38:45.152222', '0x46', '0x32', '1.234', '5.678', '9.123', '4.567', '0x98', '0x97'], ['2012-08-16 12:38:45.152519', '0x47', '0x33', '8.901', '2.345', '6.789', '0.123', '0x96', '0x95']]

I thought this was a fairly simple string stripping job, and I tried code inside the original iteration (where dmtVal was originally populated), but that didn’t work, so I took the stripping operation outside the loop, as listed above, and it is still not working. I’m thinking I’m making some kind of noob-error, but can’t find it. Any suggestions would be welcome!

---

Thanks to all of you for prompt and useful responses. Here is the corrected code:

from lxml import etree as ET
from datetime import datetime

xmlDoc = ET.parse('http://192.168.1.198/Bench_read_scalar.xml')

print '...Starting to parse XML nodes'

response = xmlDoc.getroot()

tags = (
'address',
'status',
'flow',
'dp',
'inPressure',
'actVal',
'temp',
'valveOnPercent',
)

dmtVal = []

for dmt in response.iter('dmt'):
    val = [' '.join(dmt.xpath('./%s/text()' % tag)) for tag in tags]
    val.insert(0, str(datetime.now())) #Add timestamp at beginning of each record
    dmtVal.append(val)

Which yields:

...Starting to parse XML nodes
[['2012-08-16 14:41:10.442776', '0x46', '0x32', '1.234', '5.678', '9.123', '4.567', '0x98', '0x97'], ['2012-08-16 14:41:10.443052', '0x47', '0x33', '8.901', '2.345', '6.789', '0.123', '0x96', '0x95']]
...Done

Thanks everyone!

Answer 1

The answer is: Don’t create the strings in the first place.

Your problem is in this part of the code:

for dmt in response.iter('dmt'):
    val = [str(dmt.xpath('./%s/text()' % tag)) for tag in tags]

I’m guessing you used str() here to try to extract the string from the list xpath() returns.
However, this is not what you’re getting; str() just gives you a string representation of the list.

You have a few choices to do what you want.
But given you’re parsing html, and therefor can’t know for sure how many elements the list will contain, your best option is probably using ''.join():

for dmt in response.iter('dmt'):
    val = [''.join(dmt.xpath('./%s/text()' % tag)) for tag in tags]

EDIT: You won’t need your last loop if you use this code.