Openssl::Ssl::Sslcontext Sni Servername_Cb Not Working

OpenSSL::SSL::SSLContext SNI servername_cb Not Working

Naturally you use the undocumented 'hostname' method for OpenSSL::SSLSocket!

tcp_client = TCPSocket.new("#{instance["domain"]}", 443)
ssl_context = OpenSSL::SSL::SSLContext.new()
ssl_context.ssl_version = :TLSv1
ssl_client = OpenSSL::SSL::SSLSocket.new(tcp_client, ssl_context)
ssl_client.hostname = instance["domain"]
ssl_client.connect
cert = OpenSSL::X509::Certificate.new(ssl_client.peer_cert)
ssl_client.sysclose
tcp_client.close

I discovered that here while writing code with a similar goal.

received handshake warning: unrecognized_name

Okay, I went through the source and found a way, although I'm not sure how enthusiastically I recommend it.

Although the javadoc for SSLParameters.setServerNames doesn't say so, if the value set is an empty List (with no elements), then ClientHandshaker actually sends no SNI at all. I suspect this is because the RFCs e.g. for 1.2 specify the min size as 1, prohibiting an empty list. (Compare to certificate_list in the Certificate message in TLS vs SSL; in SSL the min size was 1, and a client with no cert&key suitable for a server request didn't send the message at all, while in TLS it is 0, and a client with no suitable cert&key is explicitly specified to send a message containing an empty list.) While this is logical, since it is neither documented nor explicitly commented, I wouldn't be really happy relying on it.

Since it is pretty complicated (and fragile) to directly determine the other parameters needed, I think the best approach is to start from the existing parameters and modify, e.g. for SSLSocket:

SSLSocket s = SSLSocketFactory.getDefault() /* or other */ .createSocket("host", 443);
SSLParameters p = s.getSSLParameters();
p.setServerNames( new ArrayList<SNIServerName>() );
/* or j9+ p.setServerNames( List<SNIServerName>.of() ); */
s.setSSLParameters(p);
...

and for HttpsURLConnection, your original SSLSocketFactoryWrapper approach is quite close, except as above I would modify based on the actual parameters for the created SSLSocket and you must use the empty new ArrayList<SNIServerName>() and not .add anything to it.

Something very similar should work for Apache HttpClient, but I haven't gone through it, because I find that annoyingly like a maze of twisty little classes all alike.

PS: the source also confirms why varying sysprop jsse.enableSNIExtension won't work; that (like many others) is read and cached when JSSE is first loaded and not read subsequently. You could use reflection to break into the class and change the cached value, but let's not go there.

How can I retrieve the TLS/SSL peer certificate of a remote host using python?

The python ssl library seems like it only parses out the cert for you if it has a valid signature.

    """Returns a formatted version of the data in the
    certificate provided by the other end of the SSL channel.
    Return None if no certificate was provided, {} if a
    certificate was provided, but not validated."""

You can still get the server certificate with the ssl.get_server_certificate() function, but it returns it in PEM format. (Alternatively, you could call c.getpeercert(True), which returns the cert in binary DER format, whether it's validated or not.)

>>> print ssl.get_server_certificate(('server.test.com', 443))
-----BEGIN CERTIFICATE-----
MIID4zCCAsugAwIBA.....

From here, I would use M2Crypto or OpenSSL to read the cert and get values:

# M2Crypto
cert = ssl.get_server_certificate(('www.google.com', 443))
x509 = M2Crypto.X509.load_cert_string(cert)
x509.get_subject().as_text()
# 'C=US, ST=California, L=Mountain View, O=Google Inc, CN=www.google.com'

# OpenSSL
x509 = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
x509.get_subject().get_components()
#[('C', 'US'),
# ('ST', 'California'),
# ('L', 'Mountain View'),
# ('O', 'Google Inc'),
# ('CN', 'www.google.com')]

Boost SSL verifies expired and self-signed certificates

The issue here is Server Name Indication (SNI):

Server Name Indication (SNI) is an extension to the TLS computer
networking protocol by which a client indicates which hostname it is
attempting to connect to at the start of the handshaking process. This
allows a server to present multiple certificates on the same IP
address and TCP port number and hence allows multiple secure (HTTPS)
websites (or any other Service over TLS) to be served off the same IP
address without requiring all those sites to use the same certificate.

The badssl.com server is sending a certificate with a proper chain when you connect with no SNI. If you connect with SNI then the self-signed certificate will be sent. You can verify this with OpenSSL on the command line by observing the difference between the two commands:

openssl s_client -connect self-signed.badssl.com:443 -showcerts
openssl s_client -connect self-signed.badssl.com:443 -servername self-signed.badssl.com -showcerts

boost::asio has no API to add SNI, but I think you can do it by using the underlying OpenSSL API and the native_handle() method on your stream. It should be something like this:

SSL_set_tlsext_host_name(socket.native_handle(), "self-signed.badssl.com");

I do note that you are configuring your context with sslv3_client. As SNI is a TLS extension (i.e. not SSLv3), this may not work without configuring a TLS context.

Some clients accept SSL cert; others reject it

Some HTTP clients accept this certificate, and others do not. What could make the difference?

The short answer: load balancing, virtual hosting and SNI.

The long answer... first, here's an analysis of the certificate. We need to go though this to ensure there's no obvious mistakes.

From the dump below, there's a wildcard DNS name in the Common Name. Placing a DNS name in the CN is deprecated by both the IETF and the CA/Browser Forums. A "friendly name" should be placed in the CN because its displayed to the user. While its deprecated, its not forbidden.

Instead, DNS names should go in the Subject Alternate Name. There should be two of them. The first would be lucidpress.com and the second would be *.lucidpress.com. You need just lucidpress.com because the wildcard needs to match a label.

For reference, the IETF deprecates a DNS name in the CN in RFC 6125 Section 3.1 Server Identity; and Section 6.4.4 Checking of Common Names.

The CA/Browser Forums deprecates a DNS name in the CN in Baseline Requirements (BR) Section 9.2.2 Subject Common Name Field. Also, according to the CA/B, the Subject Alternate Name is required. See Section 9.2.1 Subject Alternative Name Extension.

Related: RFC 6125, Section 6.4.3, also does not allow the matching of *.lucidpress.com to lucidpress.com. The CA/B BR covers wildcards in Section 11.1.3, but it does not discuss matching rules.

---

With the background information above and the certificate below, here's what is going on.

You have 2 names in the default certificate. Its served by default by Apache because its the first virtual host in the configuration file.

• lucidchart.com
• *.lucidchart.com

You have 2 names in the Lucid Press' certificate.

• lucidpress.com
• *.lucidpress.com

I think the difference is Server Name Indication (SNI). Its a TLS extension, so you need TLS 1.0 or above. Those that have no trouble get the Lucid Press certifcate and use TLS 1.0 or above with SNI; those that have trouble get the default certificate and use SSLv3 or no SNI. Windows XP will use TLS 1.0 but not SNI, so its experienced often in the field due to the deployment base.

The browsers accept it because they are using TLS 1.0 or above and sending the SNI extension. Because SNI allows your Apache server to select the proper certificate during the handshake, there are no name matching problems.

Java rejects it because it uses SSLv3, even when you say SSLContext.getInstance("TLS");. You have to jump through some hoops to ensure you really get TLS 1.0 and above. There's a few questions on Stack Overflow about it. See, for example, Which Cipher Suites to enable for SSL Socket?.

Python rejects it because I'm guessing you are using 2.x, or you are allowing SSLv3. You need 3.0 or above to get SNI. See Python 3 Support? on the Python FAQ.

wget added support for SNI in version 1.14. I suspect wget is not enabling its or using SSLv3.

cURL likely ensures SNI is used if available. Daniel is very thorough, and he tries to ensure a trouble free experience and secure posture out of the box.

---

In the OpenSSL dump, the options of interest are -tls1 -servername. You can get TLS without SNI by omitting -servername. So you need both tls1 and -servername <host>.

$ openssl s_client -tls1 -servername www.lucidpress.com \
     -connect www.lucidpress.com:443 | openssl x509 -text -noout
depth=3 C = US, O = "The Go Daddy Group, Inc.", OU = Go Daddy Class 2 Certification Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 12250220837273305 (0x2b8582cd6cfed9)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certs.godaddy.com/repository/, CN=Go Daddy Secure Certificate Authority - G2
        Validity
            Not Before: May 12 16:20:34 2014 GMT
            Not After : Jul  9 22:19:45 2015 GMT
        Subject: OU=Domain Control Validated, CN=*.lucidpress.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:c8:e0:f6:77:03:c9:5e:cb:51:e3:d3:7a:b6:60:
                    d9:3d:60:26:9c:4b:00:c5:cb:b1:55:2e:d9:ee:f5:
                    08:8d:b7:64:e9:31:2e:83:e4:24:f3:89:4e:46:87:
                    b8:55:b6:34:0a:c9:3b:55:08:10:77:13:7e:85:d6:
                    8c:fa:06:dd:c1:7f:fa:9e:13:c8:1a:d8:36:22:3c:
                    cb:16:9f:cb:c7:5b:7c:7c:0b:6d:c3:ef:24:45:15:
                    5a:7a:38:dd:df:83:eb:c3:ea:9b:57:d5:8f:d8:6c:
                    ff:33:4a:21:02:2a:92:9a:e0:5d:58:51:75:07:b6:
                    ad:21:8c:34:91:20:f5:00:9e:f6:dd:90:7e:a8:60:
                    0e:14:73:de:90:a1:f4:29:83:a0:d8:9d:29:e5:de:
                    c5:cb:b5:36:84:ba:30:d4:a9:9f:b9:bf:89:26:e5:
                    80:5a:f6:3b:27:cc:6d:3f:31:1e:cc:51:09:12:73:
                    a6:de:da:b9:a4:19:86:68:7f:e6:2b:c7:3b:a6:ce:
                    6a:5a:dd:c9:ac:61:18:80:f5:d4:f1:6a:70:2c:9f:
                    8f:af:a6:c5:1d:78:97:97:90:92:6c:21:61:39:ce:
                    f8:c9:99:e2:02:b5:ce:ba:dc:f4:46:ba:e3:1f:ec:
                    ce:a5:e4:6b:56:1e:e6:20:89:44:7b:2c:9f:3a:c4:
                    33:f1
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://crl.godaddy.com/gdig2s1-59.crl

            X509v3 Certificate Policies: 
                Policy: 2.16.840.1.114413.1.7.23.1
                  CPS: http://certificates.godaddy.com/repository/

            Authority Information Access: 
                OCSP - URI:http://ocsp.godaddy.com/
                CA Issuers - URI:http://certificates.godaddy.com/repository/gdig2.crt

            X509v3 Authority Key Identifier: 
                keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE

            X509v3 Subject Alternative Name: 
                DNS:*.lucidpress.com, DNS:lucidpress.com
            X509v3 Subject Key Identifier: 
                CA:97:CC:32:09:20:3E:5F:23:05:4C:DD:F2:DA:4B:1C:E5:02:E8:69
    Signature Algorithm: sha256WithRSAEncryption
         4e:0c:8e:af:d5:c7:06:9e:b9:2c:36:97:d0:9e:1c:84:e8:e1:
         69:5a:36:a3:4f:9f:81:c9:78:5d:ca:35:df:63:be:23:88:4c:
         ba:eb:17:15:22:78:96:5d:5f:dc:3b:fa:cf:14:b6:e9:3a:fe:
         28:19:1c:85:d2:1b:23:b3:79:6d:b2:1d:76:6b:84:97:80:43:
         1b:c0:b7:14:78:75:f9:47:31:6e:21:56:0d:5e:73:ed:d3:b2:
         4b:ab:dc:b0:af:18:ee:2d:bb:65:ff:c7:cb:ff:53:64:8f:a5:
         e8:aa:45:da:fc:0f:b5:8f:da:0f:3e:b1:3b:d0:47:49:52:af:
         8d:f7:a3:42:3b:d3:a1:f4:a1:22:d5:fe:2f:4c:59:b4:18:3f:
         62:1e:4e:56:65:9b:2b:d6:76:cd:29:74:d6:74:a4:7b:bb:6f:
         b2:1d:45:12:67:14:b3:06:a7:36:ee:3a:48:d1:d6:80:2b:fa:
         6d:8b:64:01:0f:1e:51:48:0f:8b:e3:7d:13:86:79:a2:b2:04:
         05:cb:8d:07:35:d9:fa:7e:6d:5d:42:c0:a5:f4:b2:8e:57:53:
         24:b3:aa:e6:92:b1:70:07:73:98:00:91:9b:0f:3e:6e:fe:1d:
         78:7c:57:68:47:d7:8e:6f:1a:64:26:7b:69:f5:b1:13:c2:71:
         2d:ac:56:b6

---

$ dig www.lucidchart.com

; <<>> DiG 9.8.5-P1 <<>> www.lucidchart.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19608
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.lucidchart.com.        IN  A

;; ANSWER SECTION:
www.lucidchart.com. 8   IN  CNAME   chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com.
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 10 IN A 107.23.98.6
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 10 IN A 54.236.129.63
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 10 IN A 54.88.154.168

;; Query time: 23 msec
;; SERVER: 172.16.1.10#53(172.16.1.10)
;; WHEN: Sun Aug 10 00:02:52 EDT 2014
;; MSG SIZE  rcvd: 160

$ dig www.lucidpress.com

; <<>> DiG 9.8.5-P1 <<>> www.lucidpress.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34260
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.lucidpress.com.        IN  A

;; ANSWER SECTION:
www.lucidpress.com. 599 IN  CNAME   chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com.
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 59 IN A 54.88.154.168
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 59 IN A 107.23.98.6
chart-production-webserver-1858537325.us-east-1.elb.amazonaws.com. 59 IN A 54.236.129.63

;; Query time: 48 msec
;; SERVER: 172.16.1.10#53(172.16.1.10)
;; WHEN: Sun Aug 10 00:02:38 EDT 2014
;; MSG SIZE  rcvd: 160

---

If interested, this is from sslscan:

  Prefered Server Cipher(s):
    SSLv3  256 bits  DHE-RSA-AES256-SHA
    TLSv1  256 bits  DHE-RSA-AES256-SHA
    TLSv1.1  256 bits  DHE-RSA-AES256-SHA
    TLSv1.2  256 bits  DHE-RSA-AES256-GCM-SHA384

Unable to establish SSL connection, how do I fix my SSL cert?

SSL23_GET_SERVER_HELLO:unknown protocol

This error happens when OpenSSL receives something other than a ServerHello in a protocol version it understands from the server. It can happen if the server answers with a plain (unencrypted) HTTP. It can also happen if the server only supports e.g. TLS 1.2 and the client does not understand that protocol version. Normally, servers are backwards compatible to at least SSL 3.0 / TLS 1.0, but maybe this specific server isn't (by implementation or configuration).

It is unclear whether you attempted to pass --no-check-certificate or not. I would be rather surprised if that would work.

A simple test is to use wget (or a browser) to request http://example.com:443 (note the http://, not https://); if it works, SSL is not enabled on port 443. To further debug this, use openssl s_client with the -debug option, which right before the error message dumps the first few bytes of the server response which OpenSSL was unable to parse. This may help to identify the problem, especially if the server does not answer with a ServerHello message. To see what exactly OpenSSL is expecting, check the source: look for SSL_R_UNKNOWN_PROTOCOL in ssl/s23_clnt.c.

In any case, looking at the apache error log may provide some insight too.

OpenSSL for 64-bit Windows and no shared cipher

As it turns out, there was a problem with finding the certificate and private key. Problem solved.

---

Related Topics