Question

I can’t seem to get Fabric to play nice with backgrounding a process that I’ve used nohup on. . . It should be possible, given various pieces of information, including here and here.

def test():
h = 'xxxxx.compute-1.amazonaws.com'    
ports = [16646, 9090, 6666]

with settings(host_string = h):
    tun_s = "ssh  -o StrictHostKeyChecking=no -i ~/.ssh/kp.pem %s@%s " % (env.user, h)      

    for port in ports:
        p_forward = "-L %d:localhost:%d" % (port, port)
        tun_s = "%s %s" % (tun_s, p_forward)

    tun_s = "%s -N" % tun_s
    # create the tunnel. . .
    print "creating tunnel %s" % tun_s
    run("nohup '%s' >& /dev/null < /dev/null &" % tun_s)
    print "fin"

Abbreviated output:

ubuntu@domU-xxx:~/deploy$ fab test
executing on tunnel ssh  -o StrictHostKeyChecking=no -i ~/.ssh/kp.pem ubuntu@xxx  -L 16646:localhost:16646 -L 9090:localhost:9090 -L 6666:localhost:6666 -N
[xxx.compute-1.amazonaws.com] run: nohup 'ssh  -o StrictHostKeyChecking=no -i ~/.ssh/kp.pem [e-mail]  -L 16646:localhost:16646 -L 9090:localhost:9090 -L 6666:localhost:6666 -N' >& /dev/null < /dev/null &
fin

Done.
Disconnecting from xxxx

I know there is no problem with the tunnel command per se because if I strip away the nohup stuff it works fine (but obviously Fabric hangs). I’m pretty sure that it’s not properly getting detached and when the run function returns the tunnel process is immediately dying.

But why?

This also happens with a python command in another part of my code.

Answer 1

So, it seems after much wrangling that this is not possible for whatever reason with my setup (default Ubuntu installs on EC2 instances). I have no idea why and as it seems possible according to various sources.

I fixed my particular problem by using Paramiko in place of Fabric, for calls that need to be left running in the background. The following achieves this:

import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
privkey = paramiko.RSAKey.from_private_key_file('xxx.pem')
ssh.connect('xxx.compute-1.amazonaws.com', username='ubuntu', pkey=privkey)
stdin, stdout, stderr = ssh.exec_command("nohup ssh  -f -o StrictHostKeyChecking=no -i     ~/.ssh/xxx.pem [e-mail] -L 16646:localhost:16646 -L -N >& /dev/null < /dev/null &")
ssh.close()