Question

I have created a python script to try and make my life as a system administrator a lot easier.
The point of this script is to convert a Microsoft DHCP server dump file into a sorted CSV file.

I will include the code here and am thankfull for all kinds of improvements.

My problem

My script creates a list of lists (one for each dhcp reservation). For example:

[
  # [DHCP SERVER, IP ADDRESS, MAC ADDRESS, HOSTNAME, DESCRIPTION]
  [server1,172.16.0.120,31872fcefa33,wks120.domain.net,Description of client]
  [server1,172.16.0.125,4791ca3d7279,wks125.domain.net,Description of client]
  [server1,172.16.0.132,6035a71c930c,wks132.domain.net,Description of client]
  ...
]

The unused ip addresses are not listed. But I would like my script to automatically add sublists for all the unused IP addresses in between and give them a comment saying “Unregistered” or something.

I have no clue how to even begin searching google on how to complete this task, so any help would be appreciated 🙂

The script

#!/usr/bin/python
import sys, shlex
from operator import itemgetter

# Function: get_dhcp_reservations
#
# Extracts a list of ip reservations from a Microsoft DHCP server dump file
# then it stores the processed reservations them in a nested list

def get_dhcp_reservations(dmpFile):

  # Setup empty records list
  records = []

  # Open dump file for reading
  dmpFile = open(dmpFile,"r")

  # Iterate dump file line by line
  for line in dmpFile:

    # Only user lines with the word "reservedip" in it
    if "reservedip" in line:

      # Split the line into fields excluding quoted substrings
      field = shlex.split(line)

      # Create a list of only the required fields
      result = [field[2][1:9], field[7], field[8], field[9], field[10]]

      # Append each new record as a nested list
      records.append(result)

  # Return the rendered data
  return records


# Function: sort_reservations_by_ip
#
# Sorts all records by the IPv4 address field

def sort_reservations_by_ip(records):

  # Temporarily convert dotted IPv4 address to tuples for sorting
  for record in records:
    record[1] = ip2tuple(record[1])

  # Sort sublists by IP address
  records.sort(key=itemgetter(1)) 

  # Convert tuples back to dotted IPv4 addresses
  for record in records:
    record[1] = tuple2ip(record[1])

  return records


# Function: ip2tuple
#
# Split ip address into a tuple of 4 integers (for sorting)

def ip2tuple(address):
  return tuple(int(part) for part in address.split('.'))


# Function: tuple2ip
#
# Converts the tuple of 4 integers back to an dotted IPv4 address

def tuple2ip(address):

  result = ""

  for octet in address:
    result += str(octet)+"."

  return result[0:-1]


# Get DHCP reservations
records = get_dhcp_reservations(sys.argv[1])

# Sort reservations by IP address
records = sort_reservations_by_ip(records)

# Print column headings
print "DHCP Server,Reserved IP,MAC Address,Hostname,Description"

# Print in specified format records
for record in records:
  print record[0]+","+record[1]+",\""+record[2]+"\","+record[3]+","+record[4]

NOTE: I have also tried IPv4 sorting by using the python socket.inet_ntoa as suggested in other topics on this site but was not successful to get it working.

Dump File Example

Per request, here is some of the dump file

[Ommited content]

# ======================================================================
#  Start Add ReservedIp to the Scope : 172.16.0.0, Server : server1.domain.net            
# ======================================================================


    Dhcp Server \\server1.domain.net Scope 172.16.0.0 Add reservedip 172.16.0.76 0800278882ae "wks126devlin.domain.net" "Viana (VM)" "BOTH"
    Dhcp Server \\server1.domain.net Scope 172.16.0.0 Add reservedip 172.16.0.118 001e37322202 "WKS18.domain.net" "Kristof (linux)" "BOTH"
    Dhcp Server \\server1.domain.net Scope 172.16.0.0 Add reservedip 172.16.0.132 000d607205a5 "WKS32.domain.net" "Lab PC" "BOTH"
    Dhcp Server \\server1.domain.net Scope 172.16.0.0 Add reservedip 172.16.0.156 338925b532ca "wks56.domain.net" "Test PC" "BOTH"
    Dhcp Server \\server1.domain.net Scope 172.16.0.0 Add reservedip 172.16.0.155 001422a7d474 "WKS55.domain.net" "Liesbeth" "BOTH"
    Dhcp Server \\server1.domain.net Scope 172.16.0.0 Add reservedip 172.16.0.15 0800266cfe31 "xpsystst.domain.net" "Pascal (VM)" "BOTH"

[Ommited content]

Answer 1

I started by creating a list of all the empty reservations, then overwriting it with the non-empty list you started with:

#!/usr/bin/env python

reservations = [
    # [DHCP SERVER, IP ADDRESS, MAC ADDRESS, HOSTNAME, DESCRIPTION]
    ['server1','172.16.0.120','31872fcefa33','wks120.domain.net','Description of client'],
    ['server1','172.16.0.125','4791ca3d7279','wks125.domain.net','Description of client'],
    ['server1','172.16.0.132','6035a71c930c','wks132.domain.net','Description of client'],
]

def reservationlist(reservations, serverpattern, addresspattern, hostpattern,
        start, end):
    result = []
    for i in range(start, end + 1):
        result.append([
            serverpattern % i,
            addresspattern % i,
            '[no mac]',
            hostpattern % i,
            'Unregistered'])

    for reservation in reservations:
        index = int(reservation[1].split('.')[3]) - start
        result[index] = reservation

    return result

print reservationlist(
    reservations,
    'server%d',
    '172.16.0.%d',
    'wks%d.domain.net',
    120,
    132)

The final result looks like:

[['server1', '172.16.0.120', '31872fcefa33', 'wks120.domain.net', 'Description of client'],
['server121', '172.16.0.121', '[no mac]', 'wks121.domain.net', 'Unregistered'],
['server122', '172.16.0.122', '[no mac]', 'wks122.domain.net', 'Unregistered'],
['server123', '172.16.0.123', '[no mac]', 'wks123.domain.net', 'Unregistered'],
['server124', '172.16.0.124', '[no mac]', 'wks124.domain.net', 'Unregistered'],
['server1', '172.16.0.125', '4791ca3d7279', 'wks125.domain.net', 'Description of client'],
['server126', '172.16.0.126', '[no mac]', 'wks126.domain.net', 'Unregistered'],
['server127', '172.16.0.127', '[no mac]', 'wks127.domain.net', 'Unregistered'],
['server128', '172.16.0.128', '[no mac]', 'wks128.domain.net', 'Unregistered'],
['server129', '172.16.0.129', '[no mac]', 'wks129.domain.net', 'Unregistered'],
['server130', '172.16.0.130', '[no mac]', 'wks130.domain.net', 'Unregistered'],
['server131', '172.16.0.131', '[no mac]', 'wks131.domain.net', 'Unregistered'],
['server1', '172.16.0.132', '6035a71c930c', 'wks132.domain.net', 'Description of client']]

Bah! I couldn’t help myself. This version accepts IP addresses for the start and end values:

#!/usr/bin/env python

reservations = [
    # [DHCP SERVER, IP ADDRESS, MAC ADDRESS, HOSTNAME, DESCRIPTION]
    ['server1','172.16.0.120','31872fcefa33','wks120.domain.net','Description of client'],
    ['server1','172.16.0.125','4791ca3d7279','wks125.domain.net','Description of client'],
    ['server1','172.16.0.132','6035a71c930c','wks132.domain.net','Description of client'],
]

def addr_to_int(address):
    """Convert an IP address to a 32-bit int"""
    a, b, c, d = map(int, address.split('.'))
    return a * 256 * 256 * 256 + b * 256 * 256 + c * 256 + d

def int_to_addr(value):
    """Convert a 32-bit int into a tuple of its IPv4 byte values"""
    return value >> 24, value >> 16 & 255, value >> 8 & 255, value & 255

def reservationlist(reservations, serverpattern, addresspattern, hostpattern,
        start, end):

    reservationdict = dict((addr_to_int(item[1]), item)
            for item in reservations)
    startint = addr_to_int(start)
    endint = addr_to_int(end)
    for i in range(startint, endint + 1):
        try:
            item = reservationdict[i]
        except KeyError:
            addressbytes = int_to_addr(i)
            item = [
                serverpattern.format(*addressbytes),
                addresspattern.format(*addressbytes),
                '[no mac]',
                hostpattern.format(*addressbytes),
                'Unregistered']
        yield item

for entry in reservationlist(
    reservations,
    'server{3}',
    '172.16.{2}.{3}',
    'wks{3}.domain.net',
    '172.16.0.120',
    '172.16.1.132'):
    print entry

This version uses the yield keyword to turn reservationlist() into a generator. Instead of holding all the values in RAM at once, it just emits a single value at a time until the loop is finished. For each pass through the loop, it tries to fetch the actual value from your list of reservations (using a dict for quick access). If it can’t, it uses the string.format method to fill in the string templates with the IPv4 address bytes.

A quick note on address manipulation

The int_to_addr function takes a 32-bit IP address like:

AAAAAAAABBBBBBBBCCCCCCCCDDDDDDDD

and returns 4 bytes in the range 0-255, like:

AAAAAAAA, BBBBBBBB, CCCCCCCC, DDDDDDDD

In that function, >> means “rotate the value to the right that many bits”, and “& 255” means “only return the last 8 bits (128 + 64 + 32 + 16 + 8 + 4 + 2 + 1)”.

So if we’re passed in the “AAAA…DDDD” number above:

• value >> 24 => AAAAAAAA
• value >> 16 => AAAAAAAABBBBBBBB. That value & 255 => BBBBBBBB
• value >> 8 => AAAAAAAABBBBBBBBCCCCCCCC. That value & 255 => CCCCCCCC
• value & 255 => DDDDDDDD

That’s the more-or-less standard way of converting a 32-bit IPv4 address into a list of 4 bytes. When you join those values together with a dot, you get the normal “A.B.C.D” address format.