Question

For using openSSL API for public key encryption, how is the key (public & private) initialized in a C program, given private key in *.key file format, and public key in *.pem file format:

 EVP_PKEY *key;
 /* How is key initialized ?
  */
  ctx = EVP_PKEY_CTX_new(key);

Thanks.

Answer 1

try this:

        EVP_PKEY *pkey;
        FILE *f = fopen("<path for your PEM or DER encoded key>", "rb");
        if (f == NULL){
                // error handling...
        }
    //if your key is PEM encoded use this
        pkey = PEM_read_PUBKEY(f, NULL, NULL, NULL); // pkey now contains the pubKey. 
    //We are passing NULL to the others parameters because we dont need password to read a public key

    //if your key is DER encoded use this
        pkey = d2i_PUBKEY_fp(f, NULL);

        if (pkey == NULL){
                // error handling...
        }

I didnt test but should work.