Question

To begin with, I don’t know the first thing about Python … so I can really use any pointers you have. I do know some Perl, Bash scripting and a bit C++.

I’m running DenyHosts (http://denyhosts.sourceforge.net/) which every now and then sends me a message through email that an IP address was added to /etc/deny.hosts. Eg.:

Added the following hosts to /etc/hosts.deny:

87.215.133.109 (unknown)

----------------------------------------------------------------------

So far so good, but I want to add the country of the IP address to this message. To do this I created a small Perl script that spits out the country:

/usr/local/bin/geo-ip.pl --short 87.215.133.109
Netherlands

So all I want to do is to call this Perl script from Python and then fill the result in the message string. I located the source code which I suspect I need to change, but as announced at the top of this message, I don’t know the first thing about Python.

This is a sniplet from the main program calling a subroutine in report.py

deny_hosts.py:

#print deny_hosts
new_denied_hosts, status = self.update_hosts_deny(deny_hosts)
if new_denied_hosts:
    if not status:
        msg = "WARNING: Could not add the following hosts to %s" % self.__prefs.get('HOSTS_DENY')
    else:
        msg = "Added the following hosts to %s" % self.__prefs.get('HOSTS_DENY')
    self.__report.add_section(msg, new_denied_hosts)
    if self.__sync_server: self.sync_add_hosts(new_denied_hosts)
    plugin_deny = self.__prefs.get('PLUGIN_DENY')
    if plugin_deny: plugin.execute(plugin_deny, new_denied_hosts)

I think the change should go somewhere in here.
report.py defines the add_section:

def add_section(self, message, iterable):
    self.report += "%s:\n\n" % message
    for i in iterable:
        if type(i) in (TupleType, ListType):
            extra = ": %d\n" % i[1]
            i = i[0]
        else:
            extra = ""
        if self.hostname_lookup:
            hostname = self.get_hostname(i)
            debug("get_host: %s", hostname)
        else: hostname = i

        self.report += "%s%s\n" % (hostname, extra)

        if self.use_syslog:
            syslog.syslog("%s - %s%s" %(message, hostname, extra))
    self.report += "\n" + "-" * 70 + "\n"

Please help me change the code in such a way that it’ll spit out a message like:

Added the following hosts to /etc/hosts.deny:

87.215.133.109 (Netherlands, unknown)

----------------------------------------------------------------------

EDIT3:
This is how I solved it. The output is identical to the original message. After changing the sources, the daemon needs to be restarted (sudo /etc/init.d/denyhosts restart)

    def add_section(self, message, iterable):
    # added geo-ip
    # moving this from statement to the top of the file makes pycheck generate
    # a lot of errors, so I left it here.
    from subprocess import Popen, PIPE
    # end geo-ip hack import
    self.report += "%s:\n\n" % message
    for i in iterable:
        if type(i) in (TupleType, ListType):
            extra = ": %d\n" % i[1]
            i = i[0]
        else:
            extra = ""
        if self.hostname_lookup:
            hostname = self.get_hostname(i)
            debug("get_host: %s", hostname)
        else: hostname = i

        # self.report += "%s%s\n" % (hostname, extra)
        # JPH: added geo-ip
        geocmd = "/usr/local/bin/geo-ip.pl --short %s" % i
        country = Popen( geocmd, shell=True, stdout=PIPE).communicate()[0]
        country = country.strip()
        self.report += "%s%s\n%s\n" % (hostname, extra, country)
        # end geo-ip hack            

        if self.use_syslog:
            syslog.syslog("%s - %s%s" %(message, hostname, extra))
    self.report += "\n" + "-" * 70 + "\n"

Also help me understand what I change, so I can learn a bit Python today too.

EDIT2: For the sake of sharing a link to the geo-ip.pl script http://wirespeed.xs4all.nl/mediawiki/index.php/Geo-ip.pl

EDIT1: Recompilation is done automatically when the source changes, so that answers the question below.
The second problem I have with this is that I found two matching files on my system:

• /usr/share/denyhosts/DenyHosts/report.py
• /usr/share/denyhosts/DenyHosts/report.pyc

where the .py is the source code and I suspect .pyc actually being executed. So when I change the source code, I wouldn’t be surprised nothing changes if I don’t somehow compile it afterwards.

Answer 1

I’m only going to answer the specific part of your question about how to call your perl script via python and get the output. The part about where to slot in this info is a little too vague for me to guess from your snippets…

from subprocess import Popen, PIPE

hostIP = "87.215.133.109"
cmd = "/usr/local/bin/geo-ip.pl --short %s" % hostIP
output = Popen(cmd, shell=True, stdout=PIPE).communicate()[0]

## alternate form ##
# cmd = ["/usr/local/bin/geo-ip.pl, "--short", hostIP]
# output = Popen(cmd, stdout=PIPE).communicate()[0]

print output.strip() 
# Netherlands

Update

Since I am doing a few things at once on that Popen line, and you are new to python (based on your comments below), I wanted to break down that line a bit for you…

# this call to Popen actually returns a 
# Popen object with a number of methods and attributes
# to interact with the process that was just created
p = Popen(cmd, shell=True, stdout=PIPE)

# communicate() is a method of a Popen object which
# allows you to wait for the return output of the pipes
# that you named (or send data to stdin)
# It blocks until data is ready and returns a tuple (stdout, stderr)
stdout, stderr = p.communicate()

# We only wanted the stdout in this case, so we took the first index
output = p.communicate()[0]

# output is a string, and strings have the strip() method to remove 
# surrounding whitespace
stripped_output = output.strip()