Question

def execute_on_host((hostname, command), username=config['username'], keyfile=config['keyfile']):

    print hostname
    ssh_client = paramiko.SSHClient()
    ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh_client.connect(hostname=hostname, username=username, key_filename=keyfile)
    stdin, stdout, stderr = ssh_client.exec_command(command)
    print stdout.read()
    ssh_client.close()
    return stdout

So, after the hostname and stdout.read() is printed, I get errors like these

Process PoolWorker-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 99, in worker
    put((job, i, result))
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 390, in put
    return send(obj)
PicklingError: Can't pickle <type 'cStringIO.StringO'>: attribute lookup cStringIO.StringO failed

I am using the following code to execute the command.

from multiprocessing import Pool
pool = Pool(len(host_cmds_list))
pool.map(execute_on_host, host_cmds_list)
pool.close()
pool.join()

I am not sure how to fix this.

print host_cmds_list
[('hostname1', '/bin/date'), ('hostname2', '/bin/date')]

Answer 1

Your stdout is a cStringIO object which can’t be pickled. Try returning the contents of it instead of the object itself.