Question

i want to encrypt and decrypt strings in my c++ appication and use openssl for that.
since i don’t know exactly how to do it, i used this code from the internet:

        LPCTSTR encrypt(LPCTSTR inString, LPCTSTR inKey, LPCTSTR outString)
        {   
            const unsigned char* inStringC = (const unsigned char*)inString;
            const unsigned char* outStringC = (const unsigned char*)outString;
            const unsigned char* inKeyC = (const unsigned char*)inKey;

            HINSTANCE libeay32 = LoadLibrary("libeay32.dll");

            GET_FUNC_PTR(BF_set_key, void, void*, int, const unsigned char*);
            GET_FUNC_PTR(BF_cfb64_encrypt, void, unsigned char*, unsigned char*, long, void*, unsigned char*, int*, int);

            if (BF_set_key == NULL || BF_cfb64_encrypt == NULL) {
                TRACE("ERROR: failed while loading functions from \"libeay32.dll\"\n");
                return NULL;
            }

            BF_KEY key = {NULL, NULL};  
            BF_set_key(&key, strlen((const char*)inKeyC), inKeyC);

            size_t length = strlen(inString);
            unsigned char *cfb64_out = (unsigned char*)malloc((length+2)*sizeof(unsigned char*));
            unsigned char iv[32];
            memset(cfb64_out,0,length+1);
            memset(iv,0,32);
            int num = 0;    
            BF_cfb64_encrypt((unsigned char*)inStringC, cfb64_out, length, &key, iv, &num, BF_ENCRYPT); 
            FreeLibrary(libeay32);
            std::string retString = base64_encode((const char *)cfb64_out);
            strcpy((char*)outStringC, retString.c_str());
            free(cfb64_out);
            return outString;
        }

        LPCTSTR decrypt(LPCTSTR inString, LPCTSTR inKey, LPCTSTR outString)
        {
            const unsigned char* inStringC = (const unsigned char*)inString;
            const unsigned char* outStringC = (const unsigned char*)outString;
            const unsigned char* inKeyC = (const unsigned char*)inKey;

            HINSTANCE libeay32 = LoadLibrary("libeay32.dll");
            GET_FUNC_PTR(BF_set_key, void, void*, int, const unsigned char*);
            GET_FUNC_PTR(BF_cfb64_encrypt, void, unsigned char*, unsigned char*, long, void*, unsigned char*, int*, int);

            if (BF_set_key == NULL || BF_cfb64_encrypt == NULL) {
                TRACE("ERROR: failed while loading functions from \"libeay32.dll\"\n");
                return NULL;
            }


            BF_KEY key = {NULL, NULL};
            BF_set_key(&key, strlen((const char*)inKeyC), inKeyC);
            std::string retString = base64_decode((const char*)inStringC);

            size_t length = retString.length();
            unsigned char *cfb64_out = (unsigned char*)malloc((length+2)*sizeof(unsigned char));
            unsigned char iv[32];
            memset(cfb64_out,0,length+1);
            memset(iv,0,32);
            int num = 0;

            BF_cfb64_encrypt((unsigned char * )retString.c_str(), cfb64_out, length, &key, iv, &num, BF_DECRYPT);

            FreeLibrary(libeay32);

            strcpy((char *)outStringC, (char *)cfb64_out);
            free(cfb64_out);
            return outString;
        }

this works most times. but some times not.for example with the input “as” and the key “hfsa” it fails. since i am sure openssl is working i guess i did something wrong in calling the openssl functions. any ideas?

edit:
“it fails” means that either the encrypted string is empty or the decrypted string is empty. most times when it fails the decrypted string is only a substring of the expected.

edit2:
i isolated the problem to this:
if i encrypt for example “sdg” with the key “dg” then the openssl function

                BF_cfb64_encrypt((char * )inputStr, cfb64_out, length, &key, iv, &num, BF_ENCRYPT);

return “ƒx” which has the length 2.
when i decode that i have to tell the decrypt function of openssl (see above) the length to decrypt. this is 2.
but the original string had length 3. so i get only “sd” as decryption result instead of “sdg”.

Answer 1

char* works differently within byte oriented encryption than for strings. Normally it holds null terminated strings. In this case it doesn’t, it holds a byte array of definate length (3 in your case). The bytes in it can have any value including 00h, the null termination character, depending on the key, the data and the IV. So you just need to remember that (with CFB) your input lenght is your output length, and specify that particular length when decrypting (in other words, you need to communicate the length between the part that does the encryption and the part that does the decryption).