Question

Working on a script to collect users browser history with time stamps ( educational setting).
Firefox 3 history is kept in a sqlite file, and stamps are in UNIX epoch time… getting them and converting to readable format via a SQL command in python is pretty straightforward:

sql_select = """ SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch','localtime'), 
                        moz_places.url 
                 FROM moz_places, moz_historyvisits 
                 WHERE moz_places.id = moz_historyvisits.place_id
             """
get_hist = list(cursor.execute (sql_select))

Chrome also stores history in a sqlite file.. but it’s history time stamp is apparently formatted as the number of microseconds since midnight UTC of 1 January 1601….

How can this timestamp be converted to a readable format as in the Firefox example (like 2010-01-23 11:22:09)? I am writing the script with python 2.5.x ( the version on OS X 10.5 ), and importing sqlite3 module….

Answer 1

This may not be the most Pythonic code in the world, but here’s a solution: Cheated by adjusting for time zone (EST here) by doing this:

utctime = datetime.datetime(1601,1,1) + datetime.timedelta(microseconds = ms, hours =-5)

Here’s the function : It assumes that the Chrome history file has been copied from another account into /Users/someuser/Documents/tmp/Chrome/History

def getcr():
    connection = sqlite3.connect('/Users/someuser/Documents/tmp/Chrome/History')
    cursor = connection.cursor()
    get_time = list(cursor.execute("""SELECT last_visit_time FROM urls"""))
    get_url = list(cursor.execute("""SELECT url from urls"""))
    stripped_time = []
    crf = open ('/Users/someuser/Documents/tmp/cr/cr_hist.txt','w' )
    itr = iter(get_time)
    itr2 = iter(get_url)

    while True:
        try:
            newdate = str(itr.next())
            stripped1 = newdate.strip(' (),L')
            ms = int(stripped1)
            utctime = datetime.datetime(1601,1,1) + datetime.timedelta(microseconds = ms, hours =-5)
            stripped_time.append(str(utctime))
            newurl = str(itr2.next())
            stripped_url = newurl.strip(' ()')
            stripped_time.append(str(stripped_url))
            crf.write('\n')
            crf.write(str(utctime))
            crf.write('\n')
            crf.write(str(newurl))
            crf.write('\n')
            crf.write('\n')
            crf.write('********* Next Entry *********') 
            crf.write('\n')
        except StopIteration:
            break

    crf.close()            

    shutil.copy('/Users/someuser/Documents/tmp/cr/cr_hist.txt' , '/Users/parent/Documents/Chrome_History_Logs')
    os.rename('/Users/someuser/Documents/Chrome_History_Logs/cr_hist.txt','/Users/someuser/Documents/Chrome_History_Logs/%s.txt' % formatdate)