Question

I want to parse log file from below format, there are multiple urls are repetitive and I want to compute total size of each and every urls as well as compute the size of particular type of urls, what would be the best way?

/images/img81a.jpg 6620
/images/img88a.jpg 6990
/images/img80b.jpg 5909 
/images/swb-30-furniture.gif 6216 
/images/button-arrow.png 498
/images/button-arrow-down.png 484 
/images/img81a.jpg 6620 
/images/img80b.jpg 5909 
/images/back-to-top_off.gif 1506 
/images/new-logo.gif 3377 
/images/img81a.jpg 6620        

Result:

Total size computation: 11503

All particular type of urls size computation:

 /images/img81a.jpg 19860
 /images/img80b.jpg 11818
 like wise ...

I append all size values to list and perform addition to get the total size result, but for similar kind of urls size computation requires two dimension dict, I guess. Which I am not about to create.

 a['/images/img81a.jpg'][6620] = 3
 a['/images/img88a.jpg'][6990] = 1
 a['/images/img80b.jpg'][5909] = 2
 like wise ...

Answer 1

Let’s say that you have all your lines in a list:

with open('log.txt') as f:
    dico = dict()
    total_value = 0
    for line in f:
        #Feeding the dictionary
        split_array = line.split()
        possible_key = split_array[0]
        value = int(split_array[1])

        #If url has already been processed, update the value
        #Else, initialize the entry in the dictionary => default=0
        dico[possible_key] = dico.get(possible_key, default=0) + value

        #Updating the global sum
        total_value = total_value + value

Usage:

dico['/images/img81a.jpg'] => 19860