Java Client Certificates Over Https/Ssl

Java client certificates over HTTPS/SSL

Finally solved it ;). Got a strong hint here (Gandalfs answer touched a bit on it as well). The missing links was (mostly) the first of the parameters below, and to some extent that I overlooked the difference between keystores and truststores.

The self-signed server certificate must be imported into a truststore:

keytool -import -alias gridserver -file gridserver.crt -storepass $PASS -keystore gridserver.keystore

These properties need to be set (either on the commandline, or in code):

-Djavax.net.ssl.keyStoreType=pkcs12
-Djavax.net.ssl.trustStoreType=jks
-Djavax.net.ssl.keyStore=clientcertificate.p12
-Djavax.net.ssl.trustStore=gridserver.keystore
-Djavax.net.debug=ssl # very verbose debug
-Djavax.net.ssl.keyStorePassword=$PASS
-Djavax.net.ssl.trustStorePassword=$PASS

Working example code:

SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
URL url = new URL("https://gridserver:3049/cgi-bin/ls.py");
HttpsURLConnection conn = (HttpsURLConnection)url.openConnection();
conn.setSSLSocketFactory(sslsocketfactory);
InputStream inputstream = conn.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);

String string = null;
while ((string = bufferedreader.readLine()) != null) {
    System.out.println("Received " + string);
}

Java HTTPS client certificate authentication

Finally managed to solve all the issues, so I'll answer my own question. These are the settings/files I've used to manage to get my particular problem(s) solved;

The client's keystore is a PKCS#12 format file containing

1. The client's public certificate (in this instance signed by a self-signed CA)
2. The client's private key

To generate it I used OpenSSL's pkcs12 command, for example;

openssl pkcs12 -export -in client.crt -inkey client.key -out client.p12 -name "Whatever"

Tip: make sure you get the latest OpenSSL, not version 0.9.8h because that seems to suffer from a bug which doesn't allow you to properly generate PKCS#12 files.

This PKCS#12 file will be used by the Java client to present the client certificate to the server when the server has explicitly requested the client to authenticate. See the Wikipedia article on TLS for an overview of how the protocol for client certificate authentication actually works (also explains why we need the client's private key here).

The client's truststore is a straight forward JKS format file containing the root or intermediate CA certificates. These CA certificates will determine which endpoints you will be allowed to communicate with, in this case it will allow your client to connect to whichever server presents a certificate which was signed by one of the truststore's CA's.

To generate it you can use the standard Java keytool, for example;

keytool -genkey -dname "cn=CLIENT" -alias truststorekey -keyalg RSA -keystore ./client-truststore.jks -keypass whatever -storepass whatever
keytool -import -keystore ./client-truststore.jks -file myca.crt -alias myca

Using this truststore, your client will try to do a complete SSL handshake with all servers who present a certificate signed by the CA identified by myca.crt.

The files above are strictly for the client only. When you want to set-up a server as well, the server needs its own key- and truststore files. A great walk-through for setting up a fully working example for both a Java client and server (using Tomcat) can be found on this website.

Issues/Remarks/Tips

1. Client certificate authentication can only be enforced by the server.
2. (Important!) When the server requests a client certificate (as part of the TLS handshake), it will also provide a list of trusted CA's as part of the certificate request. When the client certificate you wish to present for authentication is not signed by one of these CA's, it won't be presented at all (in my opinion, this is weird behaviour, but I'm sure there's a reason for it). This was the main cause of my issues, as the other party had not configured their server properly to accept my self-signed client certificate and we assumed that the problem was at my end for not properly providing the client certificate in the request.
3. Get Wireshark. It has great SSL/HTTPS packet analysis and will be a tremendous help debugging and finding the problem. It's similar to -Djavax.net.debug=ssl but is more structured and (arguably) easier to interpret if you're uncomfortable with the Java SSL debug output.

4. It's perfectly possible to use the Apache httpclient library. If you want to use httpclient, just replace the destination URL with the HTTPS equivalent and add the following JVM arguments (which are the same for any other client, regardless of the library you want to use to send/receive data over HTTP/HTTPS):

   -Djavax.net.debug=ssl
   -Djavax.net.ssl.keyStoreType=pkcs12
   -Djavax.net.ssl.keyStore=client.p12
   -Djavax.net.ssl.keyStorePassword=whatever
   -Djavax.net.ssl.trustStoreType=jks
   -Djavax.net.ssl.trustStore=client-truststore.jks
   -Djavax.net.ssl.trustStorePassword=whatever

How do I use client certificates in a client java application?

In order to load your certificates into your application your will need to package them into a truststore.

Creating a truststore

given the 3 files:

• client-cert.pem
• client-key.pem
• root.pem

Run the following commands in your terminal. Replace PASSWORD with your desired password.

1. Package your client key and certificate into a keystore. This will create a PKCS12 keystore file.

   openssl pkcs12 -export \
       -inkey client-key.pem -in client-cert.pem \
       -out client.pfx -passout pass:PASSWORD \
       -name qlikClient
   

2. Add the keystore to your truststore. It will create a truststore if the destination doesn't exit. This will create a PKCS12 truststore file. By default it creates a JKS file which is a proprietary format. By specifying -deststoretype PKCS12 you will create a file which is in an industry standard format.

   keytool -importkeystore \
       -destkeystore truststore.pfx -deststoretype PKCS12 -deststorepass PASSWORD \
       -srckeystore client.pfx -srcstorepass PASSWORD -srcstoretype PKCS12 \
       -alias qlikClient
   

3. Add your root CA to the truststore

   keytool -importcert \
       -keystore truststore.pfx -storepass PASSWORD \
       -file root.pem -noprompt \
       -alias qlikServerCACert
   

Note that in the above commands we use the same PASSWORD for both the keystore and the truststore. You could alternatively use different passwords. Also note that you have to specify an alias for each item you add to the truststore.

If you want your truststore to trust all cacerts available in your system add -trustcacerts option to step 2 or 3.

You can use the following command to list the contents of your truststore

keytool -list -keystore truststore.pfx -storepass PASSWORD

Using the truststore in you application

Once you have your truststore you need to load it into your application. Assuming you have a constant KEYSTORE_PATH holding the path to your truststore and keyStorePass holding the password, read the truststore file into a KeyStore

private KeyStore readStore() {
  try (InputStream keyStoreStream = new FileInputStream(KEYSTORE_PATH)) {
    KeyStore keyStore = KeyStore.getInstance("PKCS12"); // or "JKS"
    keyStore.load(keyStoreStream, keyStorePass.toCharArray());
    return keyStore;
  } catch (KeyStoreException | CertificateException | NoSuchAlgorithmException e) {
    throw new RuntimeException(e);
  }
}

Create a custom SSLContext and a custom HttpClient,

final KeyStore truststore = readStore();

final SSLContext sslContext;
try {
  sslContext = SSLContexts.custom()
      .loadTrustMaterial(truststore, new TrustAllStrategy())
      .loadKeyMaterial(truststore, keyStorePass.toCharArray(), (aliases, socket) -> "qlikClient")
      .build();
} catch (NoSuchAlgorithmException | KeyStoreException | KeyManagementException | UnrecoverableKeyException e) {
  throw new RuntimeException("Failed to read keystore", e);
}
final CloseableHttpClient httpClient = HttpClients.custom().setSSLContext(sslContext).build();

You can now use this HttpClient to make requests to your API.

HttpResponse response = httpClient.execute(new HttpGet("https://sense-gcp-central1eu.net:4242/qrs/app/full"));

Or, if you are using the OpenUnirest/unirest-java library, you can configure Unirest to use your custom HttpClient

Unirest.config().httpClient(httpClient);
HttpResponse<JsonNode> response = Unirest.get("https://sense-gcp-central1eu.net:4242/qrs/app/full").asJson();

References

• https://alvinalexander.com/java/java-keytool-keystore-certificate-tutorials
• How do I use an SSL client certificate with Apache HttpClient?

Establishing connection to HTTPS using client certificates

The .jks file is the Java Keystore. It should contain the correct client certificates (and maybe also the intermediate certificates from the certificate chain).

I assume you are going to write a client that uploads the file to the HTTPS server? Then you should use the .jks file with the client certificate with the (let's say apache) HttpClient.

You need to create a SSLContext and load the keystore

SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(new File("keystore", "yourPassword".toCharArray(), new TrustSelfSignedStrategy()).build();

Then you have to put the sslContextin a SSLConnectionSocketFactory

SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslcontext, new String[] { "TLSv1" }, null, SSLConnectionSocketFactory.getDefaultHostnameVerifier());

And then finally build the HttpClient

HttpClient httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).build();

After all these steps the httpClient should use your client certificate from the keystore for your desired request.

Certificate java client https

I recently wrote a blog post, complete with examples, for doing this on Android, though the code for making the connection is the same on the desktop.

You basically have two choices - either import the certificate into a java Keystore using keytool as indicated by the post pointed to by tigran, or create the keystore in memory as needed, as in the example that accompanies my blog post.

Java client certificates and keystores

My understanding is that if you have to store a certificate with an alias matching the target domain name (in our case i.domain.io or r.domain.io) so java can provide the associated certificate as a client certificate when you are attempting a SSL connection to that domain e.g. https://r.domain.io

That's not the case at all.

Matching is based on the certificate_authorities list sent by the server in its TLS CertificateRequest message (the issuers), and on the type of key (e.g. RSA or DSA). Some imperfect matches can be chosen if the attributes are not quite as expected (see this answer), but you'd at least want to have your client certificate issued by a CA the server advertises for (this is generally done automatically on the server side when you configure the CA certificates it's willing to accept, unless you explicitly change the configuration there).

If intermediate certificates are required, you'll certainly want to make sure you've imported the full chain.

Essentially, there is no point having twice the same certificate in your keystore.

(You can try to force a specific alias by extending your own X509KeyManager, but that's not sufficient; in particular, that's not going to make the server request it nor make the chain valid.)

You need to make sure that the server is configured to request a certificate. This can sometimes be done via renegotiation, so the CertificateRequest TLS message might not necessarily be visible using Wireshark. However, you should be able to see it from the client side using -Djavax.net.debug=ssl (or all): the official documentation for Debugging SSL/TLS Connections has an example if you search for CertificateRequest on the page.

Then, you need to make sure your certificate (or the top of the chain on the client end, if there are intermediate certs) was issued by one of the CAs that is advertised in that CertificateRequest message (the Issuer DN must match).

(If the certificate authorities list in CertificateRequest is empty, but the CertificateRequest message is still sent, the client will send the first certificate it finds in its keystore by default. That sort of scenario is atypical, since it requires custom configuration on the server side in general.)

SSL socket connection with client authentication

I think you have several problems with your setup.

To configure properly the SSL connection with JSSE you need several things depending if you need to authenticate the server, the client, or to perform mutual authentication.

Let's suppose the later and more complete use case of mutual authentication.

The objective is to configure a SSLSocketFactory that you can use to contact your server.

To configure a SSLSocketFactory, you need a SSLContext.

This element in turn with require at least two elements for the mutual authentication use case, a KeyManagerFactory, required for client side SSL authentication, i.e., the server to trust the client, and TrustManagerFactory, required for configuring the client to trust the server.

Both KeyManagerFactory and TrustManagerFactory require a properly configured keystore with the necessary cryptographic material.

So, the first step will consist on generating this cryptographic material.

You already created a keystore with the server certificate:

keytool -keystore serverpublic.keystore -alias clientstore -import -file server.crt.der -storepass yourserverpublickeystorepassword

Please, be aware that, in a similar way as in the server case, you also need to create a public and private key pair for your client, of course, different than the server one.

The related code you provided with OpenSSL and keytool looks appropriate. Please, repeat the process for the client side:

openssl req -new -text -out client.csr
openssl rsa -in clientpriv.pem -out client.key
openssl req -x509 -in client.csr -text -key client.key -out client.crt
// You can use PKCS12 also with Java but it is also ok on this way
openssl pkcs12 -inkey client.key -in client.crt -export -out client.pkcs12
// Do not bother yourself and, in this use case, use always the same password for the key and keystore
keytool -importkeystore -srckeystore client.pkcs12 -srcstoretype PKCS12 -destkeystore client.keystore -storepass "yourclientkeystorepassword"

With the right keystores in place, try something like the following to interact with your server:

// First, let's configure the SSL for client authentication
KeyStore clientKeyStore = KeyStore.getInstance("JKS");
clientKeyStore.load(
  new FileInputStream("/path/to/client.keystore"),
  "yourclientkeystorepassword".toCharArray()
);

KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm()); // SunX509
kmf.init(clientKeyStore, "yourclientkeystorepassword".toCharArray());
KeyManager[] keyManagers = kmf.getKeyManagers();

// Now, let's configure the client to trust the server
KeyStore serverKeyStore = KeyStore.getInstance("JKS");
serverKeyStore.load(
  new FileInputStream("/path/to/serverpublic.keystore"),
  "yourserverpublickeystorepassword".toCharArray()
);

TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()); // SunX509
tmf.init(serverKeyStore);
TrustManager[] trustManagers = tmf.getTrustManagers();

SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, null); // You can provide SecureRandom also if you wish

// Create the SSL socket factory and establish the connection
SSLSocketFactory sf = sslContext.getSocketFactory();
SSLSocket socket = (SSLSocket)sf.createSocket(serverName, port);

// Interact with your server. Place your code here
// Please, consider the following link for alternatives approaches on how to 
// interchange information with the server:
// https://web.mit.edu/java_v1.5.0_22/distrib/share/docs/guide/security/jsse/samples/sockets/client/SSLSocketClient.java
// It also suggest the use of startHandshake explicitly if your are using PrintWriter for the reason explained in the example an in the docs:
// https://docs.oracle.com/en/java/javase/11/docs/api/java.base/javax/net/ssl/SSLSocket.html

//...

// Close the socket
socket.close();

The described approach can be extended to use, instead of sockets, higher level of abstraction components like HttpsURLConnection and HTTP clients - with the exception of Apache HttpClient that handles SSL differently - like OkHttp which, under the hood, use SSLSocketFactory and related stuff.

Please, also consider review this great article from IBM's DeveloperWorks, in addition to explain many of the point aforementioned will provide you great guidance with the generation of keystores for your client an server if necessary.

Please, also be aware that, depending on your server code, you may need to configure it to trust the provided client certificate.

According to your comments you are using a server side code similar to the one provided by Postgresql 8.1. Please, see the relevant documentation for configuring SSL in that database, if you are using some similar server side code it maybe could be of help.

Probably the best approach will be to generate a client certificate derived from the root certificate trusted by your server instead of using a self signed one.

I think that it will be also relevant for your server side SSL certificate an associated private key: first, create a root self signed certificate, your CA certificate, configure your server side C code to trust it, and then derive both client and server side SSL cryptographic material from that CA: probably it will simplify your setup and make everything work properly.

Java Https client certificate works absolutely differently than browsers or other tools

Finally the solution for this problem was found.

So preconditions for this problem to happen are:

• Java version < 7
• Apache httpclient version < 4.3.2
• SNI is enabled on the server

Root cause of the problem is in the old version of httpclient was used. Clients before the version 4.3.2 are not capable of dealing with SNI (Server Name Indication) and don't send necessary information in Client Hello packet (field name: server_name). Here is a issue link.

Basically SNI is used to run multiple virtual domains on the same ip-address using different server certificates for each domain. When SNI enabled server is getting Client Hello packet without server_name then it can't determine what certificate it has to use and therefore is not advertising the client with CA list.

Solution is to use Java version >= 7 and apache httpclient version >= 4.3.2.

Also wireshark is really helpful for debugging such issues and the filter example to use with it are posted by juhraffe:

ssl && (ip.dst == ip_or_domain_of_server || ip.src == ip_or_domain_of_server)

Also while using it don't forget to make wireshark collect packets on correct interface.

---

Related Topics