Home/ Questions/Q 6170757
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T23:05:43+00:00

I have server 1 which is generating a large amount of data, e.g there

  • 0

I have server 1 which is generating a large amount of data, e.g there are files that are constantly being updated, on the time scale of milliseconds.

I would like to get these files onto another server, using C++ or standard Linux methods.

Currently, I have been doing this by compressing the files every second and using scp to transfer them, and unpacking on the other server.

However, the latency of this is very high and I can’t break sub 1 second times with this.

Can anybody suggest the methods I can use to move the data with lower latency?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Just an idea, i don’t know if it’ll work for your particular situation:
    Write two programs. One that runs on the server on which you files are being updated, and it monitors the changes with inotify. The other program runs on the second server and maintains a tcp connection with the first one. Whenever the first program detects a change, it sends the changed part of the file to the second program, which can apply the change to its own copy of the file.

    Also, if the first server is not actually generating data for those file, but is reading it from the newtwork, it would be a good idea to just multicast the stream of data to both servers.

    • 0