Home/ Questions/Q 9102711
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-17T01:26:48+00:00

Im trying to parse an html output from a url call but i found

  • 0

Im trying to parse an html output from a url call but i found myself struggling on how i could get it working.

Im using the following code:

    import urllib2
    import base64 as b64
    import lxml.html as LH

    request = urllib2.Request('http://%s%s' % (fInput[1], fInput[2]))
    base64string = b64.encodestring('%s:%s' % (fInput[3], fInput[4])).replace('\n', '')
    request.add_header("Authorization", "Basic %s" % base64string)
    response = urllib2.urlopen(request)
    html = response.read()
    root = LH.fromstring(html)
    sibling_content = lambda x: [b.getparent().getnext().text_content() for b in root.cssselect("td b:contains('{0}')".format(x))]
    fields = ['groupList','namelist']


    for result in zip(*[sibling_content(field) for field in fields]):
        print result

The result i have when i print the output is:

('Admins', '\nme\nmyself\nirene')('guests', '\nhin\nhinself\nbacon')

The output that would fit for what i need is have an array like this, so i could insert it into a database:

['Admins', 'me','myself','nirene'],['guests', 'hin','hinself','bacon']

FOllowing, im sending the Html returned by the HTTP request:

<BODY bgcolor="#dddddd">
   <TABLE bgcolor="#dddddd" border="1">
      <TR>
         <TD valign="top"><B>MainList</B></TD>
         <TD>
            <TABLE>
               <TR>
                  <TD>
                     <TABLE bgcolor="#dddddd" border="1">
                        <TR>
                           <TD valign="top"><B>groupList</B></TD>
                           <TD>Admins</TD>
                        </TR>
                        <TR>
                           <TD valign="top"><B>namelist</B></TD>
                           <TD>
                              <TABLE>
                                 <TR>
                                    <TD>me</TD>
                                 </TR>
                                 <TR>
                                    <TD>myself</TD>
                                 </TR>
                                 <TR>
                                    <TD>irene</TD>
                                 </TR>
                              </TABLE>
                           </TD>
                        </TR>
                     </TABLE>
                     <TABLE bgcolor="#dddddd" border="1">
                        <TR>
                           <TD valign="top"><B>groupList</B></TD>
                           <TD>guests</TD>
                        </TR>
                        <TR>
                           <TD valign="top"><B>namelist</B></TD>
                           <TD>
                              <TABLE>
                                 <TR>
                                    <TD>hin</TD>
                                 </TR>
                                 <TR>
                                    <TD>hinself</TD>
                                 </TR>
                                 <TR>
                                    <TD>bacon</TD>
                                 </TR>
                              </TABLE>
                           </TD>
                        </TR>
                     </TABLE>
                  </TD>
               </TR>
            </TABLE>
         </TD>
      </TR>
   </TABLE>
</BODY>

Any ideas of how i could get this working?

Thanks in advance.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Using xpath:

    root=LH.fromstring(html)
[t.xpath('.//td[not(contains(.,"\n"))]/text()')
    for t in root.xpath('.//table[@bgcolor="#dddddd"]')]

    out:

    [['Admins', 'me', 'myself', 'irene'], ['guests', 'hin', 'hinself', 'bacon']]
    • 0