Question

The Problem:

Python’s ssl module is not complaining about a certificate, although the issuing CA is not provided in the cacert.pem file (Test Case 2. below). I use CAs extracted from Mozilla. Firefox is correctly complaining about the unknown CA (in this case the Department of Defence).

It seems as only the certificate itself get’s validated, not that the CA is known. I am running Python 2.7.1 and using ssl version OpenSSL 0.9.8r.

The Test Cases:

Check the following sites in Firefox and with the python example client below.

1. https://www.verisign.com – should work, CA is known
2. https://www.us.army.mil – should not work, as CA is not known
3. https://www.pcwebshop.co.uk – should not work, only Parallels Panel certificate

Case 2. gets validated by the python client although it shouldn’t.

Case 3. throws an Exception as expected:

routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

The Example Python Client:

CA file: http://curl.haxx.se/ca/cacert.pem (mozilla’s default CAs by curl maintainers).

Slightly modified version of http://docs.python.org/library/ssl.html#client-side-operation:

# test_ssl.py   
import socket, ssl, pprint, sys
host = sys.argv[1]

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# require a certificate from the server
ssl_sock = ssl.wrap_socket(s,
                           # http://curl.haxx.se/ca/cacert.pem
                           ca_certs="cacert.pem", 
                           cert_reqs=ssl.CERT_REQUIRED)

ssl_sock.connect((host, 443))

print repr(ssl_sock.getpeername())
print ssl_sock.cipher()
print pprint.pformat(ssl_sock.getpeercert())

# Set a simple HTTP request -- use httplib in actual code.
ssl_sock.write("""GET / HTTP/1.0\r
Host: """ + host + """\r\n\r\n""")

# Read a chunk of data.  Will not necessarily
# read all the data returned by the server.
data = ssl_sock.read()
print data

# note that closing the SSLSocket will also close the underlying socket
ssl_sock.close()

Usage:

python test_ssl.py www.verisign.com
python test_ssl.py www.us.army.mil
python test_ssl.py www.pcwebshop.co.uk

---

UPDATE:

With the help of strcat and others i could confirm this behavior is specific to:

• OSX Lion 10.7.1
• Python 2.7.1 & Python 2.6.7
• OpenSSL 0.9.8r 8 Feb 2011

I tested on two macs and several other boxes. I have the suspicion OpenSSL on mac uses a second source of CA certificates beside the file i pass to it. Maybe this makes http://www.us.army.mil a special testcase, as safari seems to trust it out of the box, too. Does anyone know other big self signed sites or how openssl on mac works?

Answer 1

The Solution is:

Through testing i could confirm, that on OSX openssl does use the system CA certificate store, even if not specified as seen in the python example above.

I used http://www.us.army.mil as a testcase as it is known to be a self signed site (http://royal.pingdom.com/2008/08/19/new-ssl-policy-in-firefox-hurting-tens-of-thousands-of-sites/). As it turns out the system CA certificates on OSX contain two DoD certificates and so Safari doesn’t complain, and also my python test client.

If i untrust those certificates in Keychan Access -> System Roots -> Certificates the python client shows the expected behavior, which confirms that python ssl / OpenSSL uses the system root certificates on OSX 10.7.1 whether specified or not. I don’t know if this is the expected behavior but it sure surprised me.