Question

I am trying to build a p2p application that requires the following, using RSA in OpenSSL:

-Encryption
-Decryption
-Generating Keys (done)
-Saving and loading keys (done)
-Saving the PUBLIC key as bytes so it can be sent over the sockets
-Loading keys from the above format

I have chosen to use the EVP functions, whatever that means. However I am having supreme difficulty finding which functions I need to use to do these things, and in what order. Official documentation of OpenSSL seems to be non-existant.

Does anyone know what functions I need to use in what order and their prototypes? Any example code lying around would also be nice.

Thanks much in advance,

twitchliquid64.

PS: This is what I have so far

#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <openssl/rsa.h>
#include <openssl/evp.h>
#include <openssl/objects.h>
#include <openssl/x509.h>
#include <openssl/err.h>
#include <openssl/pem.h>
#include <openssl/ssl.h>
#include <openssl/engine.h>
#include <openssl/rand.h>

RSA* Generate_KeyPair(void)
{
  char rand_buff[16];
  EVP_PKEY *pkey = NULL;
  RSA* r;
  char* pass = "passgdfgf";//for now

  int bits = 512;       //      512, 1024, 2048, 4096
  unsigned long exp = RSA_F4;     //      RSA_3
  OpenSSL_add_all_algorithms();

  RAND_seed(rand_buff, 16); //On linux: RAND_load_file("/dev/urandom", 1024);
  r = RSA_generate_key(bits,exp,NULL,NULL);

  if (RSA_check_key(r)!=1);;; //Check key - error out

  //Create EVP to save to file.
  pkey = EVP_PKEY_new();
  EVP_PKEY_assign_RSA(pkey, r);

  //Save private key
  FILE* fp = fopen("private.key", "w");
  PEM_write_PrivateKey(fp,pkey,EVP_aes_256_cbc(),NULL,0,NULL,pass)
  fclose(fp);

  //Save public key
  fp = fopen("public.key", "w");
  PEM_write_PUBKEY(fp, pkey);
  fclose(fp);

  return r;
}

EVP_PKEY* ReadPrivKey_FromFile(char* filename, char* pass)
{
  FILE* fp = fopen(filename, "r");
  EVP_PKEY* key = NULL;
  PEM_read_PrivateKey(fp, &key, NULL, pass);
  fclose(fp);

  return key;
}

EVP_PKEY* ReadPubKey_FromFile(char* filename)
{
  FILE* fp = fopen(filename, "r");
  EVP_PKEY* key = NULL;
  PEM_read_PUBKEY(fp, &key, NULL, NULL);
  fclose(fp);

  return key;
}

Answer 1

As said in a comment on my question:

You’ll find the example code that comes with OpenSSL more useful than the documentation. For example, documentation of encryption with RSA is shows in apps/rsa.c. It may help to work out the OpenSSL command lines to perform each function you want to do with the command line tool and then figure out what the code actually does (by inspecting it) so you can make your code do the same thing.
– David Schwartz

This example code was exactly what I needed, I advise anyone with a similar problem to consult the rsa code and header file, and also their are small use examples in the documentation as well.