I have the following issue:
>>> lines = tuple(open('/var/log/fail2ban.log', 'r'))
>>> for item in lines:
... item = item.strip('\n')
... if "fail2ban.actions:" in item and "[postfix]" in item and "Ban" in item:
... item = item.split(' ')
... print item
...
['2013-01-17', '11:03:51,752', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '87.111.253.157']
['2013-01-17', '11:10:42,612', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '37.206.77.26']
['2013-01-17', '11:23:08,674', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '37.2.185.188']
['2013-01-17', '12:40:44,997', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '37.2.185.188']
['2013-01-17', '13:28:38,006', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '194.106.26.177']
['2013-01-17', '13:43:56,959', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '70.27.53.95']
['2013-01-17', '14:42:36,601', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '95.120.42.12']
['2013-01-17', '14:45:35,147', 'fail2ban.actions:', 'WARNING', '[postfix]', 'Ban', '95.120.42.12']
I would very much like to know how I would go about filtering duplicates (item[6], the ip in this case) so that only unique values are printed.
You could create a list or set of IPs you’ve already seen, and then check the list before printing the line.
Something like this: