Home/ Questions/Q 8926381
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-15T07:57:46+00:00

I am currently using BeautifulSoup to extract HTML elements and attributes. I also would

  • 0

I am currently using BeautifulSoup to extract HTML elements and attributes.
I also would like to know the nested level of each element extracted.

For Example:

Sample HTML: 

<html>
  <head>
    <title>Element Attributes Test</title>
  </head>
  <body>
    <div id="abc">
      <ol id="def">
        <li class="testItem"> <a href="http://testpage.html">
        </li>
        <li class="testItem"> <table id="testTable">
               <tr>
                 <td>
                   <div id="testDiv">
                   </div>
                 </td>
               </tr>
             </table>
        </li>
      </ol>
    </div>  
  </body>  
</html>

I would like to get the path information for a particular element as output in the Path Column.

----------------------------------
Element | Attribute | Path
----------------------------------
html    | None      | document   
----------------------------------
head    | None      | html
----------------------------------
title   | None      | html.head
----------------------------------
body    | None      | html
----------------------------------
div     | id="abc"  | html.body
-----------------------------------
ol      | id="def"  | html.body.div
-----------------------------------
li      | class=".."| html.body.div.ol
-----------------------------------
a       | href=".." | html.body.div.ol.li
-----------------------------------
li      | class=".."| html.body.div.ol
-----------------------------------
table   | id="..."  | html.body.div.old.li
-----------------------------------
tr      | None      | html.body.div.li.table
-----------------------------------

I am able to extract Element and its associated attributes but unable to find out an appropriate way to get the path to that particular element.

How do I extract the same using BeautifulSoup?
Are there any other libraries that I can use to extract the same?

Thanks in advance.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You might take the following approach in getting a bottom up path for all the html elements

    >>> for elem in soup.findAll():
    path = '.'.join(reversed([p.name for p in elem.parentGenerator() if p]))
    print "{:10}|{:60}|{:10}".format(elem.name,elem.attrs, path)


html      |[]                                                          |[document]
head      |[]                                                          |[document].html
title     |[]                                                          |[document].html.head
body      |[]                                                          |[document].html
div       |[(u'id', u'abc')]                                           |[document].html.body
ol        |[(u'id', u'def')]                                           |[document].html.body.div
li        |[(u'class', u'testItem')]                                   |[document].html.body.div.ol
a         |[(u'href', u'http://testpage.html')]                        |[document].html.body.div.ol.li
li        |[(u'class', u'testItem')]                                   |[document].html.body.div.ol
table     |[(u'id', u'testTable')]                                     |[document].html.body.div.ol.li
tr        |[]                                                          |[document].html.body.div.ol.li.table
td        |[]                                                          |[document].html.body.div.ol.li.table.tr
div       |[(u'id', u'testDiv')]                                       |[document].html.body.div.ol.li.table.tr.td
>>>
    • 0