How to Enable Wire Logging for a Java Httpurlconnection Traffic

How to enable wire logging for a java HttpURLConnection traffic?

I've been able to log all SSL traffic implementing my own SSLSocketFactory on top of the default one.

This worked for me because all of our connections are using HTTPS and we can set the socket factory with the method HttpsURLConnection.setSSLSocketFactory.

A more complete solution that enables monitoring on all sockets can be found at http://www.javaspecialists.eu/archive/Issue169.html
Thanks to Lawrence Dol for pointing in the right direction of using Socket.setSocketImplFactory

Here is my not ready for production code:

public class WireLogSSLSocketFactory extends SSLSocketFactory {

    private SSLSocketFactory delegate;

    public WireLogSSLSocketFactory(SSLSocketFactory sf0) {
        this.delegate = sf0;
    }

    public Socket createSocket(Socket s, String host, int port,
            boolean autoClose) throws IOException {
        return new WireLogSocket((SSLSocket) delegate.createSocket(s, host, port, autoClose));
    }

    /*
    ...
    */

    private static class WireLogSocket extends SSLSocket {

        private SSLSocket delegate;

        public WireLogSocket(SSLSocket s) {
            this.delegate = s;
        }

        public OutputStream getOutputStream() throws IOException {
            return new LoggingOutputStream(delegate.getOutputStream());
        }

        /*
        ...
        */

        private static class LoggingOutputStream extends FilterOutputStream {
            private static final Logger logger = Logger.getLogger(WireLogSocket.LoggingOutputStream.class);
            //I'm using a fixed charset because my app always uses the same. 
            private static final String CHARSET = "ISO-8859-1";
            private StringBuffer sb = new StringBuffer();

            public LoggingOutputStream(OutputStream out) {
                super(out);
            }

            public void write(byte[] b, int off, int len)
                    throws IOException {
                sb.append(new String(b, off, len, CHARSET));
                logger.info("\n" + sb.toString());
                out.write(b, off, len);
            }

            public void write(int b) throws IOException {
                sb.append(b);
                logger.info("\n" + sb.toString());
                out.write(b);
            }

            public void close() throws IOException {
                logger.info("\n" + sb.toString());
                super.close();
            }
        }
    }
}

Is it possible to dump the HTTP Headers when using java.net.HttpURLConnection?

According to Sun's HttpURLConnection source there is some logging support via JUL.

This would require setting up java.util.logging with sun.net.www.protocol.http.HttpURLConnection.level=ALL.

See http://www.rgagnon.com/javadetails/java-debug-HttpURLConnection-problem.html for example.

There is also system property -Djavax.net.debug=all. But it's mainly useful for SSL debugging.

BTW, Wireshark is also a rather easy option.

Java: Display request of an HttpURLConnection before sending

You can put the HttpURLConnection in debug mode by enabling java.logging with

-Djava.util.logging.config.file=logging.properties

and put in logging.properties (by default in JRE_HOME\lib) the following property

sun.net.www.protocol.http.HttpURLConnection.level = ALL

Log all network interactions of Java application

Can't you use a logging proxy server, e.g. http://www.charlesproxy.com/ or http://fiddler2.com/? Charles Proxy can also decode AMF requests, Fiddler is the champion for SSL interception.

IIRC you can set system properties http.proxyHost and http.proxyPort to 127.0.0.1 and 8888, and java URLConnections will automatically connect via your logging proxy server, and you get a nice view of your HTTP Traffic.

Or, if the intention is to be able to replay the communication, use JMeter HTTP Proxy, this records in a format that is suitable for replay. It has also built-in support for handling cookies.

If the application uses nice web services, SoapUI also has a monitoring proxy that intercepts web service calls for which you have imported the WSDL.

How to debug everything that goes over the wire (http)

You don't specify if you have control over the client/front end or not, or if this is just over the backend.

A few immediate thoughts come to mind:

1. Use a packet sniffer/analyzer like WireShark (as already suggested). Works great, the only problem is that will probably generate significantly more data than you really want to see/analyze.

2. Use a proxy like Charles or Fiddler. These can be configured to show/log exactly what data you want/need. The disadvantage is that they are proxies (ie: have to be in the middle of your connection) and this can cause configuration difficulties if you are putting it in front of your server. Usually much easier to configure on the client end, and just push your client traffic through it. Fiddler is free, but only Windows based (alpha version available for Mac and Linux). Charles has a trial version, or otherwise is 50$ (if memory serves). I also do not know if Fiddler and/or Charles will run in headless mode.

3. Set up your own Http Proxy server and log everything through there. Not truly an ideal solution, but an option none the less.

4. Add a filter to your application to log all HTTP data coming in/out, but that gets tricky/messy as well as incoming Http request bodies are single read, so they need to be cached for multiple reads/accesses (see Http Servlet request lose params from POST body after read it once and https://stackoverflow.com/a/17129256/827480 for a good proposed solution to this issue)

Jsoup http logging

By default jsoup uses a implementation of java.net.HttpURLConnection So I suppose you need to turn on logging for that implementation (probably: sun.net.www.protocol.http.HttpURLConnection) or for java.net.

There is a system property that will enable logging for java net utils

-Djavax.net.debug=all

logging inbound and outbound HTTP(S) service traffic

Well, body is kind of excessive, but I think that for a state of the art solution, what you're looking for is tracing. Take a peek at OpenTracing and Jaeger which are solid CNCF backed choices. You can use things like Istio or Linkerd to help feeding tracing solution like Jaeger as well.

---

Related Topics