How to Debug/Display Request Sent Using Restclient

How to debug/display request sent using RestClient

You could try enabling RestClient's logging and see whether this provides any useful output:

RESTCLIENT_LOG=stdout path/to/my/program

or if you are using Rails

RESTCLIENT_LOG=stdout bundle exec passenger

replacing passenger with your choice of server. This will redirect all logging to standard output (your console).

Personally I prefer using more verbose tools when I need to inspect or troubleshoot HTTP requests.

You could try curl or wget if you prefer command-line, or one of the many browser extensions which let you perform requests easily, inspect output, save for future use, set up different environments etc. Both Postman and Advanced REST Client are good choices.

How to Inspect/Debug/View etc a request Generated by RestClient?

Since there are some lines of code referenced in the error logs, and a rescue block specifically (/lib/restclient/request.rb:427:in 'rescue in transmit'), you could open the gem and add some logging/puts/ statements in those methods to try and figure out where in the chain this is hanging up. It's not super surgical but it could get you headed in the right direction. If you are using bundler, I believe you can run bundle open restclient and then dig into the source code of the gem.

How to debug RESTful services?

Use an existing 'REST client' tool that makes it easy to inspect the requests and responses, like RESTClient.

Debugging RestEasy RestClient

If you want to be able to get the body of the response when an error occurs, I suggest you use javax.ws.rs.core.Response as the response type.

You could also go another route and handle exceptions using ExceptionMapper

How to output the generated request and response from Groovy RestClient?

Since it depends on HTTPClient, you could try enabling header and wire logging for your script.

http://blog.techstacks.com/2009/12/configuring-wire-logging-in-groovy-httpbuilder.html

http://hc.apache.org/httpcomponents-client-ga/logging.html

How do I debug a Quarkus/SmallRye client request

I used a Filter and Interceptor as an Exception Handler to solve this problem:

A Filter to print logs:

import lombok.extern.java.Log;
import org.glassfish.jersey.message.MessageUtils;

import javax.ws.rs.WebApplicationException;
import javax.ws.rs.client.ClientRequestContext;
import javax.ws.rs.client.ClientRequestFilter;
import javax.ws.rs.client.ClientResponseContext;
import javax.ws.rs.client.ClientResponseFilter;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerRequestFilter;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.ext.WriterInterceptor;
import javax.ws.rs.ext.WriterInterceptorContext;
import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URI;
import java.nio.charset.Charset;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.atomic.AtomicLong;

/**
 * Based on org.glassfish.jersey.filter.LoggingFilter
 */
@Log
public class LoggingFilter implements ContainerRequestFilter, ClientRequestFilter, ContainerResponseFilter,
                                      ClientResponseFilter, WriterInterceptor {

    private static final String NOTIFICATION_PREFIX = "* ";

    private static final String REQUEST_PREFIX = "> ";

    private static final String RESPONSE_PREFIX = "< ";

    private static final String ENTITY_LOGGER_PROPERTY = LoggingFilter.class.getName() + ".entityLogger";

    private static final String LOGGING_ID_PROPERTY = LoggingFilter.class.getName() + ".id";

    private static final Comparator<Map.Entry<String, List<String>>> COMPARATOR = (o1, o2) -> o1.getKey().compareToIgnoreCase(o2.getKey());

    private static final int DEFAULT_MAX_ENTITY_SIZE = 8 * 1024;

    private final AtomicLong _id = new AtomicLong(0);

    private final int maxEntitySize;

    public LoggingFilter() {

        this.maxEntitySize = LoggingFilter.DEFAULT_MAX_ENTITY_SIZE;
    }

    private void log(final StringBuilder b) {

        LoggingFilter.log.info(b.toString());
    }

    private StringBuilder prefixId(final StringBuilder b, final long id) {

        b.append(id).append(" ");
        return b;
    }

    private void printRequestLine(final StringBuilder b, final String note, final long id, final String method, final URI uri) {

        this.prefixId(b, id).append(LoggingFilter.NOTIFICATION_PREFIX)
                .append(note)
                .append(" on thread ").append(Thread.currentThread().getName())
                .append("\n");
        this.prefixId(b, id).append(LoggingFilter.REQUEST_PREFIX).append(method).append(" ")
                .append(uri.toASCIIString()).append("\n");
    }

    private void printResponseLine(final StringBuilder b, final String note, final long id, final int status) {

        this.prefixId(b, id).append(LoggingFilter.NOTIFICATION_PREFIX)
                .append(note)
                .append(" on thread ").append(Thread.currentThread().getName()).append("\n");
        this.prefixId(b, id).append(LoggingFilter.RESPONSE_PREFIX)
                .append(status)
                .append("\n");
    }

    private void printPrefixedHeaders(final StringBuilder b,
                                      final long id,
                                      final String prefix,
                                      final MultivaluedMap<String, String> headers) {

        for (final Map.Entry<String, List<String>> headerEntry : this.getSortedHeaders(headers.entrySet())) {
            final List<?> val = headerEntry.getValue();
            final String header = headerEntry.getKey();

            if(val.size() == 1) {
                this.prefixId(b, id).append(prefix).append(header).append(": ").append(val.get(0)).append("\n");
            }
            else {
                final StringBuilder sb = new StringBuilder();
                boolean add = false;
                for (final Object s : val) {
                    if(add) {
                        sb.append(',');
                    }
                    add = true;
                    sb.append(s);
                }
                this.prefixId(b, id).append(prefix).append(header).append(": ").append(sb.toString()).append("\n");
            }
        }
    }

    private Set<Map.Entry<String, List<String>>> getSortedHeaders(final Set<Map.Entry<String, List<String>>> headers) {

        final TreeSet<Map.Entry<String, List<String>>> sortedHeaders = new TreeSet<>(LoggingFilter.COMPARATOR);
        sortedHeaders.addAll(headers);
        return sortedHeaders;
    }

    private InputStream logInboundEntity(final StringBuilder b, InputStream stream, final Charset charset) throws IOException {

        if(!stream.markSupported()) {
            stream = new BufferedInputStream(stream);
        }
        stream.mark(this.maxEntitySize + 1);
        final byte[] entity = new byte[this.maxEntitySize + 1];
        final int entitySize = stream.read(entity);
        b.append(new String(entity, 0, Math.min(entitySize, this.maxEntitySize), charset));
        if(entitySize > this.maxEntitySize) {
            b.append("...more...");
        }
        b.append('\n');
        stream.reset();
        return stream;
    }

    @Override
    public void filter(final ClientRequestContext context) throws IOException {

        final long id = this._id.incrementAndGet();
        context.setProperty(LoggingFilter.LOGGING_ID_PROPERTY, id);

        final StringBuilder b = new StringBuilder();

        this.printRequestLine(b, "Sending client request", id, context.getMethod(), context.getUri());
        this.printPrefixedHeaders(b, id, LoggingFilter.REQUEST_PREFIX, context.getStringHeaders());

        if(context.hasEntity()) {
            final OutputStream stream = new LoggingFilter.LoggingStream(b, context.getEntityStream());
            context.setEntityStream(stream);
            context.setProperty(LoggingFilter.ENTITY_LOGGER_PROPERTY, stream);
            // not calling log(b) here - it will be called by the interceptor
        }
        else {
            this.log(b);
        }
    }

    @Override
    public void filter(final ClientRequestContext requestContext, final ClientResponseContext responseContext)
    throws IOException {

        final Object requestId = requestContext.getProperty(LoggingFilter.LOGGING_ID_PROPERTY);
        final long id = requestId != null ? (Long) requestId : this._id.incrementAndGet();

        final StringBuilder b = new StringBuilder();

        this.printResponseLine(b, "Client response received", id, responseContext.getStatus());
        this.printPrefixedHeaders(b, id, LoggingFilter.RESPONSE_PREFIX, responseContext.getHeaders());

        if(responseContext.hasEntity()) {
            responseContext.setEntityStream(this.logInboundEntity(b, responseContext.getEntityStream(),
                    MessageUtils.getCharset(responseContext.getMediaType())));
        }

        this.log(b);
    }

    @Override
    public void filter(final ContainerRequestContext context) throws IOException {

        final long id = this._id.incrementAndGet();
        context.setProperty(LoggingFilter.LOGGING_ID_PROPERTY, id);

        final StringBuilder b = new StringBuilder();

        this.printRequestLine(b, "Server has received a request", id, context.getMethod(), context.getUriInfo().getRequestUri());
        this.printPrefixedHeaders(b, id, LoggingFilter.REQUEST_PREFIX, context.getHeaders());

        if(context.hasEntity()) {
            context.setEntityStream(
                    this.logInboundEntity(b, context.getEntityStream(), MessageUtils.getCharset(context.getMediaType())));
        }

        this.log(b);
    }

    @Override
    public void filter(final ContainerRequestContext requestContext, final ContainerResponseContext responseContext)
    throws IOException {

        final Object requestId = requestContext.getProperty(LoggingFilter.LOGGING_ID_PROPERTY);
        final long id = requestId != null ? (Long) requestId : this._id.incrementAndGet();

        final StringBuilder b = new StringBuilder();

        this.printResponseLine(b, "Server responded with a response", id, responseContext.getStatus());
        this.printPrefixedHeaders(b, id, LoggingFilter.RESPONSE_PREFIX, responseContext.getStringHeaders());

        if(responseContext.hasEntity()) {
            final OutputStream stream = new LoggingFilter.LoggingStream(b, responseContext.getEntityStream());
            responseContext.setEntityStream(stream);
            requestContext.setProperty(LoggingFilter.ENTITY_LOGGER_PROPERTY, stream);
            // not calling log(b) here - it will be called by the interceptor
        }
        else {
            this.log(b);
        }
    }

    @Override
    public void aroundWriteTo(final WriterInterceptorContext writerInterceptorContext)
    throws IOException, WebApplicationException {

        final LoggingFilter.LoggingStream stream = (LoggingFilter.LoggingStream) writerInterceptorContext.getProperty(LoggingFilter.ENTITY_LOGGER_PROPERTY);
        writerInterceptorContext.proceed();
        if(stream != null) {
            this.log(stream.getStringBuilder(MessageUtils.getCharset(writerInterceptorContext.getMediaType())));
        }
    }

    private class LoggingStream extends FilterOutputStream {

        private final StringBuilder b;

        private final ByteArrayOutputStream baos = new ByteArrayOutputStream();

        LoggingStream(final StringBuilder b, final OutputStream inner) {

            super(inner);

            this.b = b;
        }

        StringBuilder getStringBuilder(final Charset charset) {
            // write entity to the builder
            final byte[] entity = this.baos.toByteArray();

            this.b.append(new String(entity, 0, Math.min(entity.length, LoggingFilter.this.maxEntitySize), charset));
            if(entity.length > LoggingFilter.this.maxEntitySize) {
                this.b.append("...more...");
            }
            this.b.append('\n');

            return this.b;
        }

        @Override
        public void write(final int i) throws IOException {

            if(this.baos.size() <= LoggingFilter.this.maxEntitySize) {
                this.baos.write(i);
            }
            this.out.write(i);
        }

    }

}

Using the filter in the rest client interface:

@Path("/")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
@RegisterRestClient
@RegisterProvider(LoggingFilter.class)
public interface Api {

    @GET
    @Path("/foo/bar")
    FooBar getFoorBar();

Now the request and response payload is printed in log.

After that, an interceptor to handle exceptions:

Qualifier:

@InterceptorBinding
@Target({TYPE, METHOD})
@Retention(RUNTIME)
public @interface ExceptionHandler {

}

Interceptor:

@Interceptor
@ExceptionHandler
public class ExceptionHandlerInterceptor  {

    @AroundInvoke
    public Object processRequest(final InvocationContext invocationContext) {

        try {
            return invocationContext.proceed();

        }
        catch (final WebApplicationException e) {

            final int status = e.getResponse().getStatus();
            final String errorJson = e.getResponse().readEntity(String.class);

            final Jsonb jsonb = JsonbBuilder.create();

            //"ErrorMessageDTO" is waited when a error occurs
            ErrorMessage errorMessage = jsonb.fromJson(errorJson, ErrorMessage.class);

            //isValid method verifies if the conversion was successful
            if(errorMessage.isValid()) {
                return Response
                        .status(status)
                        .entity(errorMessage)
                        .build();
            }

            errorMessage = ErrorMessage
                    .builder()
                    .statusCode(status)
                    .statusMessage(e.getMessage())
                    .success(false)
                    .build();

            return Response
                    .status(status)
                    .entity(errorMessage)
                    .build();
        }
        catch (final Exception e) {

            e.printStackTrace();

            return Response
                    .status(Status.INTERNAL_SERVER_ERROR)
                    .entity(ErrorMessage
                            .builder()
                            .statusCode(Status.INTERNAL_SERVER_ERROR.getStatusCode())
                            .statusMessage(e.getMessage())
                            .success(false)
                            .build())
                    .build();
        }
    }

}

Using the interceptor:

@Path("/resource")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
@ExceptionHandler
@Traced
@Log
public class ResourceEndpoint {

    @Inject
    @RestClient
    Api api;

    @GET
    @Path("/latest")
    public Response getFooBarLatest() {

        return Response.ok(this.api.getFoorBar()).build();
    }

ErrorMessage bean:

@RegisterForReflection
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Data
@Builder
public class ErrorMessage  {

    @JsonbProperty("status_message")
    private String statusMessage;

    @JsonbProperty("status_code")
    private Integer statusCode;

    @JsonbProperty("success")
    private boolean success = true;

    @JsonbTransient
    public boolean isValid() {

        return this.statusMessage != null && !this.statusMessage.isEmpty() && this.statusCode != null;
    }

}

PS: Using Lombok!

Spring RestTemplate - how to enable full debugging/logging of requests/responses?

I finally found a way to do this in the right way.
Most of the solution comes from
How do I configure Spring and SLF4J so that I can get logging?

It seems there are two things that need to be done :

1. Add the following line in log4j.properties : log4j.logger.httpclient.wire=DEBUG
2. Make sure spring doesn't ignore your logging config

The second issue happens mostly to spring environments where slf4j is used (as it was my case).
As such, when slf4j is used make sure that the following two things happen :

1. There is no commons-logging library in your classpath : this can be done by adding the exclusion descriptors in your pom :

           <exclusions><exclusion>
               <groupId>commons-logging</groupId>
               <artifactId>commons-logging</artifactId>
           </exclusion>
       </exclusions>
   

2. The log4j.properties file is stored somewhere in the classpath where spring can find/see it. If you have problems with this, a last resort solution would be to put the log4j.properties file in the default package (not a good practice but just to see that things work as you expect)

---

Related Topics