Question

I am trying to collect system stats using python. Mostly looking for a libery or github project that does this already. I am looking to be able to collect/log the following things.

• CPU Usage and or Load average
• Disk Free space
• Disk IO thoughput
• Ram usage (Free, Active, Cached)
• Network IO
• Bandwidth Usaged
• Uptime
• Anything else useful in server monitoring.

Edit:
I am mostly looking for support in Linux systems I will move on to windows systems later.

Answer 1

Try using the psutil module here, take a look at the example shown. However, for DISK IO you might have to use the wmi and the pywin32 modules.

For example, for DISK QUEUE LENGTH with windows use this:

from win32pdh import *
import time

while True:
    path = MakeCounterPath((None,"PhysicalDisk","_Total",None,0,"Avg. Disk Queue Length"),0)
    query_handle = OpenQuery()
    counter_handle = AddCounter(query_handle, path)
    query_handle = OpenQuery()
    counter_handle = AddCounter(query_handle, path)
    CollectQueryData(query_handle)
    time.sleep(2)
    CollectQueryData(query_handle)
    (counter_type, value) = GetFormattedCounterValue(counter_handle, PDH_FMT_DOUBLE)
    print value
    CloseQuery(query_handle)

It uses the counters of the perfmon app on windows. For linux, there are multiple tools for DISK IO.

For linux DISK IO check out iotop here

Also, example of Linux diskstats use this, it periodically parses /proc/diskstats

def diskstats_parse(dev=None):
    file_path = '/proc/diskstats'
    result = {}

    # ref: http://lxr.osuosl.org/source/Documentation/iostats.txt
    columns_disk = ['m', 'mm', 'dev', 'reads', 'rd_mrg', 'rd_sectors',
                    'ms_reading', 'writes', 'wr_mrg', 'wr_sectors',
                    'ms_writing', 'cur_ios', 'ms_doing_io', 'ms_weighted']

    columns_partition = ['m', 'mm', 'dev', 'reads', 'rd_sectors', 'writes', 'wr_sectors']

    lines = open(file_path, 'r').readlines()
    for line in lines:
        if line == '': continue
        split = line.split()
        if len(split) == len(columns_disk):
            columns = columns_disk
        elif len(split) == len(columns_partition):
            columns = columns_partition
        else:
            # No match
            continue

        data = dict(zip(columns_disk, split))
        if dev != None and dev != data['dev']:
            continue
        for key in data:
            if key != 'dev':
                data[key] = int(data[key])
        result[data['dev']] = data

    return result

I hope this helps. 🙂