We want to add automatic software updates to our application, but our company isn’t yet ready to buy a code-signing certificate from a trusted root CA, so we’ll be using a self-signed certificate to sign code updates (.exe and .dll) for now.
Question: how to verify a binary signed with a self-signed certificate, without having to install the certificate, using Microsoft’s Cryptography API? The .cer file to check against will be bundled with the application. Or is it simpler to use a generic Crypto library?
You can skip the whole X509 thing, after all you don’t really need it if you’re going to be using your own certificates…
For what you want to do, first you have to generate your RSA private/public key pair. Then you store the public key in your application.
When you have an update, you sign it on your site, by getting the MD5 or SHA-1 or whatever hash you want to use; then you encrypt that hash with the private key. The installed applications fetch the update and the signature (the encrypted hash); when the application gets the binary file, it computes its hash, then decrypts the other one using the public key and compares them. If they’re identical then it’s a valid update, otherwise you reject it and warn the user or something.
With X509 certificates that are self-signed the mechanism is going to be exactly that, but the public key is going to have a bunch of additional data like the identity of the issuer which will be the same identity of the certificate.