I’m encrypting a text (just a string) using OpenSSL’s CMS library. I’ve implemented a cms encrypt method and when I call this in a C++ project (with the equal certificate and input string) it produces the following output:
MIME-Version: 1.0
Content-Disposition: attachment; filename="smime.p7m"
Content-Type: application/pkcs7-mime; smime-type=enveloped-data; name="smime.p7m"
Content-Transfer-Encoding: base64
MIAGCSqGSIb3DQEHA6CAMIACAQAxggG9MIIBuQIBADCBoDCBkjELMAkGA1UEBhMC
QVQxDzANBgNVBAgMBlN0eXJpYTENMAsGA1UEBwwER3JhejEKMAgGA1UECgwBLzEK
MAgGA1UECwwBLzEdMBsGA1UEAwwUQ2hyaXN0b2YgU3Ryb21iZXJnZXIxLDAqBgkq
hkiG9w0BCQEWHXN0cm9tYmVyZ2VyQHN0dWRlbnQudHVncmF6LmF0AgkAmHFnJtIY
YyAwDQYJKoZIhvcNAQEBBQAEggEACLskYA0ma3hBccwOamh14/b2XqRCmBakGxPM
dQFMoiQy47UvGLQ4QmruOU1Mv530r3jglxVZd2DNX5fBPwHJ91ORU39BGns2BnWd
E5z8yH5Kr1edjErj/EZRzJFU1Qyq6/uBn3W4X9+jNhuWWcPrxoQOoQhrE0vETnv4
dZb5ic1iYLWOraSwnQmvOLgrh9iCJuq6n9EWF/YHJelETKQSO2RnPvbpesHLgZ48
ngGkDH+FWU0QZV+LXmq8xpdpLWxMAeh07WIUz0sA1okYFMCk2uy5sg7ovyO804ae
AbZlXz8aDeoMMGzOfNi2PxYxbwRwObBOj2cxU0qMQu49lgIhJjCABgkqhkiG9w0B
BwEwFAYIKoZIhvcNAwcECNUojhuQn568oIAEGPkzqWrziObAHieBNpIKMGboxxY8
oiTMIAQIToaGyI0IMGcAAAAAAAAAAAAA
But when I copy the code into a objective-c Project (for iOS5) I get a “wrong” output which I can’t decrypt anymore… This is the output:
MIME-Version: 1.0
Content-Disposition: attachment; filename="smime.p7m"
Content-Type: application/pkcs7-mime; smime-type=enveloped-data; name="smime.p7m"
Content-Transfer-Encoding: base64
MIAGCSqGSIb3DQEHA6CAMIACAQAxggG4MIIBtAIBADCBoDCBkjELMAkGA1UEBhMC
QVQxDzANBgNVBAgMBlN0eXJpYTENMAsGA1UEBwwER3JhejEKMAgGA1UECgwBLzEK
MAgGA1UECwwBLzEdMBsGA1UEAwwUQ2hyaXN0b2YgU3Ryb21iZXJnZXIxLDAqBgkq
hkiG9w0BCQEWHXN0cm9tYmVyZ2VyQHN0dWRlbnQudHVncmF6LmF0AgkAmHFnJtIY
YyAwDQYJKoZIhvcNAQEBBQAEgfwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwgAYJKoZIhvcNAQcBMBQG
CCqGSIb3DQMHBAgAyfDfER+rUaCABBi7ammjNh3zr0CZDxRjalXcmxC5qIbWsoUE
COCbSOGWOGcWAAAAAAAAAAAAAA==
It’s exactly the same code but produces two different outputs. To me the second output seems very strange because of the mane “AAAA” in base64 encoding. I guess this must be the part of the certificate. At the bottom is the encrypted input string and the begin of the certificate is equal but it changes strangely in the middle or end of the certificate.
Does anyone have a suggestion what’s going wrong here?
Okay… the problem was a bug in the OpenSSL lib. When compiling for a certain architecture (i.e. i386) and using it in an x64 environment the
CMS_encrypt()method produces the aforementioned wrong output. This bug results from some performance optimizations in assembler.For further information see:
http://www.openssl.org/support/faq.cgi => 12. Why does OpenBSD-i386 build fail on des-586.s with “Unimplemented segment type”?
I solved it by compiling OpenSSL with
no-asmattribute../Configure ... no-asmI had this problem because the library used for the iOS simulator needs a i386 architecture but Mac OSX uses x64. Simply compile it with
no-asmand it works.