I’ve recently extended a Python API for the Windows Azure storage APIs (PyAzure) to include support for the service management APIs. See https://github.com/bmb/pyazure.
I’m using a HTTPSClientAuthHandler like the one suggested in using pyOpenSSL to create urllib custom opener. On Linux, with various versions of Python 2.6 and 2.7 this works well. However, Windows is another story. All requests against the Azure management host address fail with:
[Errno 10054] An existing connection was forcibly closed by the remote host
Which I think, is the socket errno 10054 “Connection reset by peer”, in drag.
This doesn’t appear to be a problem in my API code (unless the client cert authentication method I’m using is bogus somehow), but something lower-level. I can reproduce the issue without urllib2 or httplib by simply setting up an SSL socket and sending the same HTTP request down the pipe as urllib2 would, e.g. to list the valid Azure data centre locations:
>>> import socket, ssl, sys
>>> sys.version
'2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit (AMD64)]'
>>> s = ssl.wrap_socket(socket.socket(), certfile='c:\\users\\blair\\research\\clouds\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> s.connect(('management.core.windows.net',443))
>>> s.send("GET /SUBSCRIPTION_ID/locations HTTP/1.1\r\nAccept-Encoding: identity\r\nX-Ms-Version: 2011-10-01\r\nHost: management.core.windows.net\r\nConnection: close\r\nUser-Agent: Python-urllib/2.6\r\n\r\n")
202
>>> s.read()
Traceback (most recent call last):
c:\Users\blair\research\clouds\azure\pyazure\<ipython-input-63-3306c981d8a7>
in <module>()
----> 1 s.read()
C:\Python27\lib\ssl.pyc in read(self, len)
136
137 try:
--> 138 return self._sslobj.read(len)
139 except SSLError, x:
140 if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
error: [Errno 10054] An existing connection was forcibly closed by the remote host
Replace SUBSCRIPTION_ID above, with your Azure subscription ID. The exception is raised ~45s after calling SSLSocket.read. The cert is a properly formatted PEM file including both the private key and certificate, it was converted from the pfx (in Ubuntu 10.04) using:
openssl pkcs12 -in pfxfile -out pemfile -nodes
I don’t think it matters here, but I also tried unix2dos-ing the PEM file, to no avail though. I get the same behaviour even when I don’t provide any cert, but doing that on Linux results in a proper API error from the server:
‘HTTP/1.1 403 Forbidden\r\nContent-Length: 0\r\nServer: Microsoft-HTTPAPI/2.0\r\nDate: Thu, 01 Dec 2011 13:59:29 GMT\r\nConnection: close\r\n\r\n’
This has been independently verified by another person using Windows 7 (same as me). It’s not a client-side firewall issue – the same code works in a NAT-ed Linux VM running on the same host.
I’m stumped. Would really appreciate any help folks here might be able to provide…
Update:
This appears to be related to the underlying SSL implementation in Python. CPython 2.7.1 has the error behaviour as shown above, but I’ve since tested and had success with ActiveState Python (both 2.7 and 2.6), e.g.:
>>> import sys, socket, ssl
>>> sys.version
'2.7.1 (r271:86832, Feb 7 2011, 11:30:38) [MSC v.1500 32 bit (Intel)]'
>>> s = ssl.wrap_socket(socket.socket(), certfile='\\\\VBOXSVR\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> s.connect(('management.core.windows.net',443))
>>> s.send('GET /SUBSCRIPTION_ID/locations HTTP/1.1\r\nAccept-Encoding: identity\r\nX-Ms-Version: 2011-10-01\r\nHost: management.core.windows.net\r\nUser-Agent: Python-urllib/2.6\r\n\r\n')
183
>>> s.read(4096)
'HTTP/1.1 200 OK\r\nContent-Length: 908\r\nContent-Type: application/xml; charset=utf-8\r\nServer: Microsoft-HTTPAPI/2.0\r\nx-ms-request-id: 08ca048cda6b445da6b3a8f3e4890197\r\nDate: Fri, 02 Dec 2011 03:02:14 GMT\r\n\r\n<Locations xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Location><Name>Anywhere US</Name><DisplayName>Anywhere US</DisplayName></Location><Location><Name>South Central US</Name><DisplayName>South Central US</DisplayName></Location><Location><Name>North Central US</Name><DisplayName>North Central US</DisplayName></Location><Location><Name>Anywhere Europe</Name><DisplayName>Anywhere Europe</DisplayName></Location><Location><Name>North Europe</Name><DisplayName>North Europe</DisplayName></Location><Location><Name>West Europe</Name><DisplayName>West Europe</DisplayName></Location><Location><Name>Anywhere Asia</Name><DisplayName>Anywhere Asia</DisplayName></Location><Location><Name>Southeast Asia</Name><DisplayName>Southeast Asia</DisplayName></Location><Location><Name>East Asia</Name><DisplayName>East Asia</DisplayName></Location></Locations>'
And as expected my API works too:
ActivePython 2.6.7.20 (ActiveState Software Inc.) based on
Python 2.6.7 (r267:88850, Jun 27 2011, 13:20:48) [MSC v.1500 64 bit (AMD64)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyazure import pyazure
>>> pa = pyazure.PyAzure(subscription_id=SUBSCRIPTION_ID, management_cert_path='c:\\users\\blair\\research\\clouds\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> list(pa.wasm.list_locations())
['Anywhere US', 'South Central US', 'North Central US', 'Anywhere Europe', 'North Europe', 'West Europe', 'Anywhere Asia', 'Southeast Asia', 'East Asia']
The Lib\ssl.py files in CPython2.7 and ActivePython2.7 are identical, so I guess this must be due to some difference in the underlying C libs, perhaps a bug in CPython. Any gurus out there?
I haven’t been able to pin down a definitive explanation for this, but after a bit of trial and error I’m confident of where the issue lays…
Short answer: it’s the ssl implementation in the http://www.python.org/ Windows bundle. Use ActiveState Python instead.
Long Answer:
The Windows CPython distributions available from http://www.python.org/download/ bundle quite an old version of OpenSSL (0.9.8l), compared with the ActiveState Python distributions which are based on CPython but (amongst other things) provide regular updates to 3rd party inclusions such as OpenSSL (currently 0.9.8r).
I downloaded Windows binaries of OpenSSL and tested via the openssl s_client interface, e.g.:
The current version works, as expected. Unfortunately it seems difficult to get one’s hands on old OpenSSL binaries for Windows, perhaps not surprising given it is a security library… But anyway, I built 0.9.8l from source under Ubuntu 10.04 and found that it hangs after sending a HTTP request down the pipe, presumably the server silently dropped the connection for some reason:
Under newer and even slightly older (e.g., Ubuntu10.04’s 0.9.8e) OpenSSLs the server responds to the request with the expected:
But with OpenSSL 0.9.8l I get nothing.