Question

i try to create a filter to a large log file like

Fri Oct  5 00:55:45 2012 [pid 2766] CONNECT: Client "157.82.250.217"
Fri Oct  5 00:55:45 2012 [pid 2765] [ftp] OK LOGIN: Client "157.82.250.217", anon password "[e-mail]"
Fri Oct  5 00:56:05 2012 [pid 2767] [ftp] FAIL DOWNLOAD: Client "157.82.250.217", "/pub/10.5524/100001_101000/100036/Gene_catalogue/Gene_catalogue.pep", 1638400 bytes, 81.81Kbyte/sec
Fri Oct  5 00:57:27 2012 [pid 3056] CONNECT: Client "157.82.250.217"
Fri Oct  5 00:57:27 2012 [pid 3055] [ftp] OK LOGIN: Client "157.82.250.217", anon password "-wget@"

I want to put the ip address in to http://www.ip-adress.com to get the organization name
like 157.82.250.217 –> University of Tokyo then add University of Tokyo behind ip address

The website is http://www.ip-adress.com/ip_tracer/157.XX.xxx.xx
and the source code like :

    <th>ISP of this IP [<a href="/isp" target="_self">?</a>]:</th>

    <td>

      University of Tokyo</td>

    </tr>

  <tr class="odd">

  <th>Organization:</th>

 <td>

  University of Tokyo</td>

  </tr>

  <tr class="even">

I’m newcomer in python, i write some for filter time, and can you help me add function inside

import time
f= open("/opt/CLiMB/Storage1/log/vsftp.log")
def OnlyRecent(line):
if  time.strptime(line.split("[")[0].strip(),"%a %b %d %H:%M:%S %Y")>  time.gmtime(time.time()-(60*60*24*7)): 
    return True
return False
filename= time.strftime('%Y%m%d')+'.log'
f1= open(filename,'w')
for line in f:
if OnlyRecent(line):
        print line
        f1.write(line)
f.close()
f1.close()

Thanks!!

Answer 1

I (usually nobody at StackOverflow) will write your code. Have a look at http://docs.python.org/library/urllib2.html. Build your url, pass it to urllib2.urlopen and read the result into a string. Then extract the data you want to have. That’s it!

In your case simple string operations should be enough to extract the data, but you could also check http://www.crummy.com/software/BeautifulSoup/.