Question

I am trying to write a Python 3 script where I am querying a web api and receiving an XML response. The response looks like this –

<?xml version="1.0" encoding="UTF-8"?>
<ipinfo>
   <ip_address>4.2.2.2</ip_address>
   <ip_type>Mapped</ip_type>
   <anonymizer_status/>
   <Network>
      <organization>level 3 communications  inc.</organization>
      <OrganizationData>
     <home>false</home>
         <organization_type>Telecommunications</organization_type>
         <naics_code>518219</naics_code>
     <isic_code>J6311</isic_code>
      </OrganizationData>      
      <carrier>level 3 communications</carrier>
      <asn>3356</asn>
      <connection_type>tx</connection_type>
      <line_speed>high</line_speed>
      <ip_routing_type>fixed</ip_routing_type>
      <Domain>
         <tld>net</tld>
         <sld>bbnplanet</sld>
      </Domain>
   </Network>
   <Location>
      <continent>north america</continent>
      <CountryData>
         <country>united states</country>
         <country_code>us</country_code>
         <country_cf>99</country_cf>
      </CountryData>
      <region>southwest</region>
      <StateData>
         <state>california</state>
         <state_code>ca</state_code>
         <state_cf>88</state_cf>
      </StateData>
      <dma>803</dma>
      <msa>31100</msa>
      <CityData>
         <city>san juan capistrano</city>
         <postal_code>92675</postal_code>
         <time_zone>-8</time_zone>
         <area_code>949</area_code>
         <city_cf>77</city_cf>
      </CityData>
      <latitude>33.499</latitude>
      <longitude>-117.662</longitude>
   </Location>
</ipinfo>

This is the code I have so far –

import urllib.request
import urllib.error 
import sys
import xml.etree.ElementTree as etree

…

try:
    xml = urllib.request.urlopen(targetURL, data=None)
except urllib.error.HTTPError as e:
    print("HTTP error: " + str(e) + " URL: " + targetURL)
    sys.exit()

tree = etree.parse(xml)
root = tree.getroot()

The API query works and through the debugger I can see all of the information inside the ‘root’ variable. My issue is that I have not been able to figure out how to extract something like the ASN (<asn></asn>) from the returned XML. I’ve been beating my head against this for a day with a whole wide variety of finds, findalls and all other sorts of methods but not been able to crack this. I think I have reached the point where I cannot see the wood for all the trees and every example I have found on the internet doesn’t seem to help. Can someone show me a code snippet which can extract the contents of a XML element from inside the tree structure?

Many thanks

Tim

Answer 1

I would recommend using Beautiful Soup.

It’s a very powerful when it comes to extracting data from xml-code.

Example:

from bs4 import BeautifulSoup
soup = BeautifulSoup(targetURL)

soup.find_all('asn') #Would return all the <asn></asn> tags found!