Question

I have a python script that output/prints a large array (50MB in size) to the screen. This script is running on machineA. Is it at all possible to ‘pipe’ this output to machineB, where it will be processed?

I know I can save the output in a file on machineA and then send the file to machineB. I was just wondering if it is possible to do so without having to save the data first to a file (on machineA).

Any solutions (python, bash shell, other ideas) will be appreciated.

Answer 1

This is a work for nc (netcat)… on machineB you do

nc -l 12345 | processing_program

This command will start netcat in “listen mode” waiting for a connection. Once the connection is established what is coming from the network will be sent to stdout and what is sent to stdin will be sent back to whoever connected.

After that you go on machineA and run

generating_program | nc machineB 12345

this will instruct netcat to start a connection to machineB (port 12345) and send whatever it gets from stdin over the wire. Whatever comes back from that connection is sent to standard output.