Let’s say I have a standalone windows service running in a windows server machine. How to make sure it is highly available?
1). What are all the design level guidelines that you can propose?
2). How to make it highly available like primary/secondary, eg., the clustering solutions currently available in the market
3). How to deal with cross-cutting concerns in case any fail-over scenarios
If any other you can think of please add it here ..
Note:
The question is only related to windows and windows services, please try to obey this rule 🙂
To keep the service at least running you can arrange for the Windows Service Manager to automatically restart the service if it crashes (see the Recovery tab on the service properties.) More details are available here, including a batch script to set these properties – Restart a windows service if it crashes
High availability is more than just keeping the service up from the outside – the service itself needs to be built with high-availabiity in mind (i.e. use good programming practices throughout, appropriate datastructures, pairs resource aquire and release), and the whole stress-tested to ensure that it will stay up under expected loads.
For idempotent commands, tolerating intermittent failures (such as locked resources) can be achieved by re-invoking the command a certain number of times. This allows the service to shield the client from the failure (up to a point.) The client should also be coded to anticipate failure. The client can handle service failure in several ways – logging, prompting the user, retrying X times, logging a fatal error and exiting are all possible handlers – which one is right for you depends upon your requirements. If the service has “conversation state”, when service fails hard (i.e. process is restarted), the client should be aware of and handle ths situation, as it usually means current conversation state has been lost.
A single machine is going to be vulnerable to hardware failure, so if you are going to use a single machine, then ensure it has redundant components. HDDs are particularly prone to failure, so have at least mirrored drives, or a RAID array. PSUs are the next weak point, so redundant PSU is also worthwhile, as is a UPS.
As to clustering, Windows supports service clustering, and manages services using a Network Name, rather than individual Computer names. This allows your client to connect to any machine running the service and not a hard-coded name. But unless you take additional measures, this is Resource failover – directing requests from one instance of the service to another. Converstaion state is usually lost. If your services are writing to a database, then that should also be clustered to also ensure reliabiity and ensure changes are available to the entire cluster, and not just the local node.
This is really just the tip of the iceberg, but I hope it gives you ideas to get started on further research.
Microsoft Clustering Service (MSCS)