Question

Python 2.4.x (cannot install any non-stock modules).

Question for you all. (assuming use of subprocess.popen)

Say you had 20 – 30 machines – each with 6 – 10 files on them that you needed to read into a variable.

Would you prefer to scp into each machine, once for each file (120 – 300 SCP commands total), reading each file after it’s SCP’d down into a variable – then discarding the file.

Or – SSH into each machine, once for each file – reading the file into memory. (120 – 300 ssh commands total).

?

Unless there’s some other way to grab all desired files in one shot per machine (files are named YYYYMMDD.HH.blah – range would be given 20111023.00 – 20111023.23). – reading them into memory that I cannot think of?

Answer 1

Depending on the size of the file, you can possibly do something like:

...
files= "file1 file2 ..."
myvar = ""
for tm in machine_list
         myvar = myvar+ subprocess.check_output(["ssh", "user@" + tm, "/bin/cat " + files]);
...

file1 file2 etc are space delimited. Assuming all are unix boxes you can /bin/cat them all in one shot from each machine. (This is assuming that you are simply loading the ENTIRE content in one variable) variations of above.. SSH will be simpler to diagnose.

At least that’s my thought.

UPDATE

use something like

         myvar = myvar+Popen(["ssh", "user@" +tm ... ], stdout=PIPE).communicate()[0]

Hope this helps.