Question

I’m sorry if this question is a bit basic, or if I’ve missed an answer elsewhere, but please take pity on a lost and confused beginner and give me a hand if you can.
I’ve got a load of long files consisting of lines of ‘date time id number random crap’ and I’m trying to count the amount of times certain numbers occur alongside certain ids. As far as I can tell, everything is right except I only get one number returned, rather than hundreds.
I’ve really cut this down to just the bits that might be going wrong. There are probably hundreds of quicker and easier ways to do what I’m trying to do, but I don’t know them yet. Please do let me know!
I have a list of the data from the file, called data, and a list of the ids present in the file.

#get date(data[i][0]), time(data[i][1]), number(data[i][3]), id(data[i][4]) from original data if a certain id (found in listofids) is present, and write into new list
data0=[]
data1=[]
etc
values=[data[i][0], data[i][1], data[i][3], data[i][4]]
for line in data:
    if listofids[0] in line:
        data0.append(values)
    if listofids[1] in line:
        data1.append(values)
    etc

#put number into list h if it occurs in list data0
h=[]
r=range (0, len(data0))
for i in r:
    number=data0[i][3]
    if number not in h:
        h.append(number)
print (len(h))
print (h)

#count the time each element in list h occurs in list data0
print (data0.count(h[0]), data0.count(h[1]), data0.count(h[2]))

I’m sorry if this is a bit messy and confusing, and I’m especially sorry if I’m missing something really easy.

Thank you for helping!

Answer 1

They’re all ’07/11/2008 09:00:06 word #0F 0006E7895B word word’ in the original file

Parsing that fixed format should be quite simple. In the following snippet I assume you want to count combinations of (ID, number):

from collections import defaultdict

# Count numbers in a dictionary that defaults to zero if a key does not exist yet
counter = defaultdict(int)

with open("filename", "rU") as f:
    for line in f:
        info = line.split()
        date, time, unused, number, id = info[:5]
        counter[id, number] += 1

for (id, number), count in counter.items():
    print("The combination id=%s, number=%s occurred %d times" % (id, number, count))