I am currently working on a VPS panel that uses a master-slave model. One master server runs a panel written in PHP, and manages multiple slave servers via SSH. The slave servers are accessed via a limited account, that can sudo to specific server administration-related commands, and all interaction is logged in a directory that the account itself does not have access to.
I am currently using PHP-SSH2, but this approach has a few problems:
- Exit codes are not reliably returned, so all commands have to be executed in a wrapper script that packages up stdout, stderr, and the exit code into a JSON object and returns it via stdout. This script has to exist on every slave server.
- The PHP-SSH2 library does not know the concept of a “custom connection timeout”, which means I have to probe a server with fsockopen before trying to use PHP-SSH2 to connect – if I don’t do that, an unreachable server may delay the pageload for a minute or more. This is even more worse because of the next issue.
- Persistent connections are not possible. This causes absolutely ridiculous pageload times in the panel, especially combined with the previous issue with timeouts.
Right now, I’m trying to solve the last problem primarily.
There are several possible solutions I have run across, but all of them have an issue of some sort:
- Using PHPSecLib, a pure PHP SSH implementation, and replacing all fsockopen calls with pfsockopen calls. This will make the connections persistent, but it’s hackier than I’d like and the security implications of persistent sockets in PHP are unclear.
- Setting up a persistent SSH tunnel from the master server to each slave server, and running a simple daemon (bound to localhost) on each slave server that runs whatever it’s told to. This is a problem for two reasons. First off it introduces the need for a daemon on the slave servers, which is something I’d rather avoid. The second issue is that if someone were to compromise a limited account on a slave server, they could still run certain system commands simply by connecting to the “command daemon”, even if they would not have access to those commands from their own shell. This is a problem.
- Running a daemon on the master server that manages persistent SSH connections to slave servers on behalf of the panel. This would involve writing an SSH client in Python (this is the only suitable language I am familiar with), and would probably come down to using paramiko. Since the documentation of paramiko is poor, this isn’t a very attractive option and may even cause security issues because it isn’t entirely clear to me how things are supposed to be used in paramiko.
The following are not an option:
- Switching to a different language for the panel itself. I am writing the panel in PHP because that is the language I am most familiar with, and I’m aware of the quirks and potential issues I might encounter. Writing an important public-facing project like this in a language I am not familiar with would be a bad idea.
- Using Twisted for the third “possible solution”. Twisted is a very large and convoluted dependency, and the documentation seems even worse than that of paramiko.
- Running a HTTPd or otherwise non-SSH public-facing daemon on the slave servers.
In practice, I am seeing pageload times of sometimes over a minute when multiple servers have to be contacted for a pageload. This is obviously not acceptable for a VPS panel.
My goal is to have some kind of implementation that avoids the connection overhead that is introduced when using PHP-SSH2. What would be the best way to do this, in a secure manner, while introducing a minimal amount of dependencies on the slave servers?
You could use autossh, and create reverse (portforward) tunnels with autossh. Then let your php application talk against those reverse ssh ports. If the ssh connection fails, autossh will keep trying to recreate the connection. Your php app will fail to connect to the reverse tunnel and not even timeout.