Validate Ssl Certificates with Python

Validate SSL certificates with Python

From release version 2.7.9/3.4.3 on, Python by default attempts to perform certificate validation.

This has been proposed in PEP 467, which is worth a read: https://www.python.org/dev/peps/pep-0476/

The changes affect all relevant stdlib modules (urllib/urllib2, http, httplib).

Relevant documentation:

https://docs.python.org/2/library/httplib.html#httplib.HTTPSConnection

This class now performs all the necessary certificate and hostname checks by default. To revert to the previous, unverified, behavior ssl._create_unverified_context() can be passed to the context parameter.

https://docs.python.org/3/library/http.client.html#http.client.HTTPSConnection

Changed in version 3.4.3: This class now performs all the necessary certificate and hostname checks by default. To revert to the previous, unverified, behavior ssl._create_unverified_context() can be passed to the context parameter.

Note that the new built-in verification is based on the system-provided certificate database. Opposed to that, the requests package ships its own certificate bundle. Pros and cons of both approaches are discussed in the Trust database section of PEP 476.

How to validate server's ssl certificate in python?

Rather than configuring your server to present a self-signed certificate, you should use a self-signed certificate as a certificate authority to sign the server certificate. (How to do this is beyond the scope of your question, but I'm sure you can find help on Stack Overflow or elsewhere.)

Now you must configure your client to trust your certificate authority. In python (2.7.9 or later), you can do this using the ssl module:

import ssl

...  # create socket

ctx = ssl.create_default_context(cafile=path_to_ca_certificate)
sslsock = ctx.wrap_socket(sock)

You can then transmit and read data on the secure socket. See the ssl module documentation for more explanation.

The urllib2 API is simpler:

import urllib2

resp = urllib2.urlopen(url, cafile=path_to_ca_certificate)
resp_body = resp.read()

If you wish to use Requests, according to the documentation you can supply a path to the CA certificate as the argument to the verify parameter:

resp = requests.get(url, verify=path_to_ca_certificate)

Python SSL certificate verify error

So the issue might have three resolutions as I see it:

1. A certificate is OK and there is something wrong with the code. The issue may occur, for example, while using prepared requests as described in this solution

   But I don't really think it is your case because in the snippet you've provided no such methods are used. For the two next variants, you'll need to get the URL that causes an error and explore it's certificate (can be done via browser).

2. A certificate is OK but a certificate authority that signed it is not included in CA list that is utilized by requests library. After you'll open a troubling URL, check CA in it and see if it's dates are valid and it is included in this list. If not, add CA in the trusted list for the requests library -- as explained in the answers to this StackOverflow question.

3. A certificate is not valid or self-singed. Same solution as in 2.

The general solution is to wrap your script in the try except clause and to print out all the URLs that will result in mistakes. Then try to request them one by one via requests library and see if the issue occurs. If it does, it's (2) or (3) case. If not — try to run script on another machine with fresh installed python and requests. If the run will be successful — then there's some issue in your configuration.

urllib and SSL: CERTIFICATE_VERIFY_FAILED Error

If you just want to bypass verification, you can create a new SSLContext. By default newly created contexts use CERT_NONE.

Be careful with this as stated in section 17.3.7.2.1

When calling the SSLContext constructor directly, CERT_NONE is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time you would like to ensure the authenticity of the server you’re talking to. Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED.

But if you just want it to work now for some other reason you can do the following, you'll have to import ssl as well:

input = input.replace("!web ", "")      
url = "https://domainsearch.p.mashape.com/index.php?name=" + input
req = urllib2.Request(url, headers={ 'X-Mashape-Key': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' })
gcontext = ssl.SSLContext()  # Only for gangstars
info = urllib2.urlopen(req, context=gcontext).read()
Message.Chat.SendMessage ("" + info)

This should get round your problem but you're not really solving any of the issues, but you won't see the [SSL: CERTIFICATE_VERIFY_FAILED] because you now aren't verifying the cert!

To add to the above, if you want to know more about why you are seeing these issues you will want to have a look at PEP 476.

This PEP proposes to enable verification of X509 certificate signatures, as well as hostname verification for Python's HTTP clients by default, subject to opt-out on a per-call basis. This change would be applied to Python 2.7, Python 3.4, and Python 3.5.

There is an advised opt out which isn't dissimilar to my advice above:

import ssl

# This restores the same behavior as before.
context = ssl._create_unverified_context()
urllib.urlopen("https://no-valid-cert", context=context)

It also features a highly discouraged option via monkeypatching which you don't often see in python:

import ssl

ssl._create_default_https_context = ssl._create_unverified_context

Which overrides the default function for context creation with the function to create an unverified context.

Please note with this as stated in the PEP:

This guidance is aimed primarily at system administrators that wish to adopt newer versions of Python that implement this PEP in legacy environments that do not yet support certificate verification on HTTPS connections. For example, an administrator may opt out by adding the monkeypatch above to sitecustomize.py in their Standard Operating Environment for Python. Applications and libraries SHOULD NOT be making this change process wide (except perhaps in response to a system administrator controlled configuration setting).

If you want to read a paper on why not validating certs is bad in software you can find it here!

---

Related Topics