I’d like to allow a group of processes to be able to communicate with each other. The processes can come-and-go and there is no obvious “main” or “server” process and so the processes really need to be able to communicate in a completely peer-to-peer basis.
I understand interprocess communication and the different communication methods (TCP, named pipes etc…) but I don’t really understand how a peer-to-peer cluster works, in particular how each client efficiently determines the list of other clients.
Is there a way of achieving this in .Net using existing technology? (e.g. WPF)
Failing that, where should I look for information on peer-to-peer communication protocols?
A study of the BitTorrent protocol for distributed peer-to-peer coordination would probably be a good place to start: http://www.bittorrent.org/beps/bep_0003.html
Other systems to investigate might be the Tor distributed anonymity network, and the Windows Internet Name Service (WINS). These all involve peer discovery and coordination to some degree.
Probably the trickiest part of setting up a P2P cluster is figuring out a reliable way for a new peer to discover the peer cluster. Using a predefined TCP port number would be a start, but falls apart when that port is already in use by some other process. You could use a UDP broadcast to find out if any peers are out there, but that won’t make it across your first router hop since routers normally filter broadcasts to prevent broadcast propagation storms. You could use a centralized tracker/dispatcher, but that becomes a single point of failure for the entire peer cluster.
Network topology will also affect your discovery approach. If all the peers will be within the same network segment, or behind the same firewall, you can do just about anything. If some of the peers are behind a firewall, though, you can’t initiate a connection to them through the firewall – they have to initiate a connection to you, or open an incoming port in the firewall using UPNP.
Discovery is the weakest part of the BitTorrent system. Unless the user knows what torrent directory or tracker URL to use, the torrent client is useless. Once you find a node, any node, in a peer cluster, finding the rest of the members of the cluster is relatively simple.
If your collection of processes all reside on the same physical machine, you could use something like a message queue service to act as a third party arbiter between transient processes. Otherwise, you’re probably looking at having one of the processes in the peer group take a leadership position to answer discovery requests and distribute the peer list to others. When the lead process needs to go, it can hand off the lead to one of the other peers, or the peers can fail over to a new lead process using the distributed peer list. That’s essentially what the Windows Name Service (WINS) does for NetBios name discovery.