Question

I am a novice programmer in python. I’m currently constructing a class that parses logfiles and have completed all elements of the class. Nonetheless, as most things in python have gone for me I have either incorrectly formatted my class or have screwed up my semantics. I was wondering if there was a definitive format for constructing classes and whether or not the one I have written, follows said format.

Here are some lines from the log:

2012-06-12 14:02:21,813 [main]  INFO  ConnectionManager.java (line 238) Initializing the ConnectionManager.
2012-06-12 14:02:21,844 [main]  INFO  CimListener.java (line 142) Starting listener at http://127.0.0.1:7012
2012-06-12 14:02:21,974 [main]  INFO  CimListener.java (line 158) Listening at http://127.0.0.1:7012
2012-06-12 14:02:23,209 [main]  INFO  RmiServiceExporter.java (line 393) Looking for RMI registry at port '10099'
2012-06-12 14:02:23,232 [main]  INFO  RmiServiceExporter.java (line 404) Could not detect RMI registry - creating new one

And here is the class:

import re
import time
import calendar
from datetime import datetime

f = open("C:\Users\-----\Desktop\Real Logs\controllersvc.log", "r")
while True:
    line = f.readline()
    if not line:
        break

class LogLine:

    SEVERITIES = ['EMERG','ALERT','CRIT','ERR','WARNING','NOTICE','INFO','DEBUG']
    severity = 1


    def __init__(self, line):
        try:
            timestr, msstr, sevstr, self.filename, linestr, self.message = re.match(r"^(\d\d\d\d-\d\d-\d\d[ \t]\d\d:\d\d:\d\d),(\d\d\d),(?i[a-z]+),([A-Za-z]{1,.}),([(]\[lL]ine\>\s+\d+[)]),^(?<=\)\s?\w+$)", line).groups()
            self.line = int(linestr)
            self.sev = self.SEVERITIES.index(sevstr)
            self.time = float(calendar.timegm(time.strptime(timestr, "%Y-%m-%d %H:%M:%S,%f"))) + float(msstr)/1000.0
            dt = datetime.strptime(t, "%Y-%m-%d %H:%M:%S,%f")
        except Exception:
            print 'error',self.filename

    def get_time(self):
        return self.time
    def get_severity(self):
        return self.sev
    def get_message(self):
        return message
    def get_filename(self):
        return filename
    def get_line(self):
        return line

Answer 1

There are several things wrong with your code:

• the self.filename attribute is not necessarily defined before it is used in init
• you should try to be more specific as to which exceptions you are expecting to catch (is a KeyboardInterruptException something you want to prohibit?)
• your regular expression is complex and hard to maintain (and I think wrong)
• you are exposing “getters” to public attributes on an object, which are not necessary.
• some of the “getters” return global variables (message, line and filename) which were presumably intended to return the instance attributes
• newly written classes should ideally subclass something (in this case, object)

I hope the list is helpful and seen as as constructive criticism.

To help you see some more pythonic code, I have written a log reader for your data, which has no complex regular expression, nor any “getters” (which you could add if you really desire). There are further simplifications one could make (use of namedtuple for instance), but in the interest of clarity and education, I have kept things vanilla:

import datetime


class LogEntry(object):
    @staticmethod
    def from_str(line):
        """Converts a log line in a string, into a LogEntry."""
        # split the line by one or more spaces
        date, time, _, severity, filename, _, line, message = re.split('\s+', line, 7)

        # strip the trailing bracket on the line and cast to an int
        line = int(line[:-1])

        # combine the date and time strings and turn them into a datetime
        dt = datetime.datetime.strptime(date + time, "%Y-%m-%d%H:%M:%S,%f")

        return LogEntry(dt, severity, filename, line, message)

    def __init__(self, dt, severity, filename, line_num, message):
        self.datetime = dt
        self.severity = severity
        self.filename = filename
        self.line_num = line_num
        self.message = message

    def __str__(self):
        return '%s %s %s L%s: %s' % (self.datetime, self.severity, self.filename, self.line_num, self.message)


if __name__ == '__main__':

    log_contents = """2012-06-12 14:02:21,813 [main]  INFO  ConnectionManager.java (line 238) Initializing the ConnectionManager.
    2012-06-12 14:02:21,844 [main]  INFO  CimListener.java (line 142) Starting listener at http://127.0.0.1:7012
    2012-06-12 14:02:21,974 [main]  INFO  CimListener.java (line 158) Listening at http://127.0.0.1:7012
    2012-06-12 14:02:23,209 [main]  INFO  RmiServiceExporter.java (line 393) Looking for RMI registry at port '10099'
    2012-06-12 14:02:23,232 [main]  INFO  RmiServiceExporter.java (line 404) Could not detect RMI registry - creating new one"""

    # uncomment for real log reading
    #fh = file(filename, 'r')

    # emulate a log file by providing an iterable of lines (just like fh will do)
    fh = log_contents.split('\n')

    for line in fh:
        print LogEntry.from_str(line.strip())

Which produces the output:

2012-06-12 14:02:21.813000 INFO ConnectionManager.java L238: Initializing the ConnectionManager.
2012-06-12 14:02:21.844000 INFO CimListener.java L142: Starting listener at http://127.0.0.1:7012
2012-06-12 14:02:21.974000 INFO CimListener.java L158: Listening at http://127.0.0.1:7012
2012-06-12 14:02:23.209000 INFO RmiServiceExporter.java L393: Looking for RMI registry at port '10099'
2012-06-12 14:02:23.232000 INFO RmiServiceExporter.java L404: Could not detect RMI registry - creating new one

I really hope this helps you discover how much fun Python can be!