Question

I am trying to write a pycurl script to access a secured site (HTTPS).

c = pycurl.Curl()
c.setopt(pycurl.USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20100101 Firefox/8.0')
c.setopt(pycurl.URL, 'https://for-example-securedsite')
c.setopt(pycurl.COOKIEFILE, 'cookie.txt')
c.setopt(pycurl.COOKIEJAR, 'cookies.txt')
c.setopt(pycurl.WRITEDATA, file("page.html","wb"))   

I am getting the below error..

pycurl.error: (60, ‘SSL certificate problem, verify that the CA cert is OK. Details:\nerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed’)

The code failed, as it failed to get the SSL cert.

The error went away if I add the below lines to my code.

c.setopt(pycurl.SSL_VERIFYPEER, 0)   
c.setopt(pycurl.SSL_VERIFYHOST, 0)

The above code will skip the certificate verification. But its subjected to ‘man in middle’ attack.

I know I have the SSL certificate in my local certificate store. Do anyone know how to export my certificate and use it my code.. Some sample codes will be awesome..

Thanks for your time!

Answer 1

You are right, the way you are doing it subjects you to a man-in-the-middle attack, especially in light of the most recent SSL vulnerabilities. You can resolve it as follows:

import pycurl
curl = pycurl.Curl()
curl.setopt(pycurl.URL, "https://your-secure-website.com/")
curl.setopt(pycurl.SSL_VERIFYPEER, 1)
curl.setopt(pycurl.SSL_VERIFYHOST, 2)
curl.setopt(pycurl.CAINFO, "/path/to/updated-certificate-chain.crt")
curl.perform()

curl by default comes with an outdated certificate list. Whether you want to update it or just use your own certs for testing, make sure to place the updated-certificate-chain.crt file in an accessible location and use the pycurl.CAINFO option to point to it.

Also make sure pycurl.SSL_VERIFYHOST is set to 2, the highest security check setting.