What Is Recommended Way for Spawning Threads from a Servlet in Tomcat

What is recommended way for spawning threads from a servlet in Tomcat

In a barebones servletcontainer like Tomcat or Jetty, your safest bet is using an applicaton wide thread pool with a max amount of threads, so that the tasks will be queued whenever necessary. The ExecutorService is very helpful in this.

Upon application startup or servlet initialization use the Executors class to create one:

executor = Executors.newFixedThreadPool(10); // Max 10 threads.

Then during servlet's service (you could ignore the result for the case that you aren't interested, or store it in the session for later access):

Future<ReturnType> result = executor.submit(new YourTask(yourData));

Where YourTask must implement Runnable or Callable and can look something like this, whereby yourData is just your data, e.g. populated with request parameter values (just keep in mind that you should absolutely not pass Servlet API artifacts such as HttpServletRequest or HttpServletResponse along!):

public class YourTask implements Runnable {

    private YourData yourData;

    public YourTask(YourData yourData) {
        this.yourData = yourData;
    }

    @Override
    public void run() {
        // Do your task here based on your data.
    }
}

Finally, during application's shutdown or servlet's destroy you need to explicitly shutdown it, else the threads may run forever and prevent the server from properly shutting down.

executor.shutdownNow(); // Returns list of undone tasks, for the case that.

In case you're actually using a normal JEE server such as WildFly, Payara, TomEE, etc, where EJB is normally available, then you can simply put @Asynchronous annotation on an EJB method which you invoke from the servlet. You can optionally let it return a Future<T> with AsyncResult<T> as concrete value.

@Asynchronous
public Future<ReturnType> submit() {
    // ... Do your job here.

    return new AsyncResult<ReturnType>(result);
}

Is it safe for a Java servlet to spawn threads in order to satisfy a request?

First, it looks you're modifying the result object in both threads. This is not thread safe because what the first and second threads do might not be visible to each other or to the thread the servlet is running on. See this article for more info.

Second, if you are modifying the response in these other threads, no, this will not be safe. Once you exit the doGet method, you should consider the response sent. In your example, there's a chance the response will get sent back to the client before those two threads have run.

Suppose the MyResult result affects the response object (you're either adding the result to the response, it's affecting the response code, etc). There are a few ways to handle this.

Use ExecutorService and Future:

public void doGet(HttpServletRequest request, HttpServletResponse response) {
   // Creating a new ExecutorService for illustrative purposes.
   // As mentioned in comments, it is better to create a global 
   // instance of ExecutorService and use it in all servlets. 
   ExecutorService executor = Executors.newFixedThreadPool(2);

   Future<Result1> f1 = executor.submit(new Callable<Result1>() {
      @Override
       public Result1 call() throws Exception {
          // do expensive stuff here.
          return result;
      }
   });

   Future<Result2> f2 = executor.submit(new Callable<Result2>() {
      @Override
      public Result2 call() throws Exception {
         // do expensive stuff here.
         return result;
      }
   });

   // shutdown allows the executor to clean up its threads. 
   // Also prevents more Callables/Runnables from being submitted.
   executor.shutdown();

   // The call to .get() will block until the executor has
   // completed executing the Callable.
   Result1 r1 = f1.get();
   Result2 r2 = f2.get();
   MyResult result = new MyResult();
   // add r1 and r2 to result.
   // modify response based on result
}

A more advanced technique is Asynchronous Processing. Using async processing is a good idea if your requests take a long time to process. It does not improve the latency of any one request, but it does allow Tomcat to handle more requests at any given point in time.

A simple example would be:

@WebServlet(urlPatterns={"/asyncservlet"}, asyncSupported=true)
// Rather than @WebServlet, you can also specify these parameters in web.xml    
public class AsyncServlet extends HttpServlet {
   @Override
   public void doGet(HttpServletRequest request, HttpServletResponse response) {
      response.setContentType("text/html;charset=UTF-8");
      final AsyncContext acontext = request.startAsync();
      acontext.start(new Runnable() {
         public void run() {
            // perform time consuming steps here.
            acontext.complete();
   }
}

Correct way to create task threads in tomcat at startup

Please refer to the following Q&A: tomcat 6 thread pool for asynchronous processing.

Also, instead of Executors.newFixedThreadPool you'll probably need Executors.newScheduledThreadPool to create instance of ScheduledExecutorService which is capable of executing repeating tasks.

Hope this helps...

Can i spawn a thread from a servlet?

There is such thing as parent and child threads, but you don't have a lot of control on that. For example there's InheritableThreadLocal, where you can store variables for a thread hierarchy.
you can spawn a new thread from a servlet. Prefer Java 5 executors framework
if using servlet 3.0, take a look at its asynchronous processing capabilities.

Threading and Concurrency Within A Servlet

Nonsense.

The JEE spec has lots of "should nots" and "thou shant's". The Servlet spec, on the other hand, has none of that. The Servlet spec is much more wild west. It really doesn't dive in to the actual operational aspects like the JEE spec does.

I've yet to see a JEE container (either a pure servlet container ala Tomcat/Jetty, or full boat ala Glassfish/JBoss) that actually prevented me from firing off a thread on my own. WebSphere might, it's supposed to be rather notorious, but I've not used WebSphere.

If the concept of creating unruly, self-managed threads makes you itch, then the full JEE containers internally have a formal "WorkManager" that can be used to peel threads off of. They just all expose them in different ways. That's the more "by the book-ish" mechanism for getting a thread.

But, frankly, I wouldn't bother. You'll likely have more success using the Executors out of the standard class library. If you saturate your system with too many threads and everything gets out of hand, well, that's on you. Don't Do That(tm).

As to whether an async solution is even appropriate, I'll punt on that. It's not clear from your post whether it is or not. But your question was about threads and Servlets.

Just Do It. Be aware it "may not be portable", do it right (use an Executor), take responsibility for it, and the container won't be the wiser, nor care.

Threads inside webapplication Java

threading in a managed environment is generally a bad idea. why not use some sort of abstraction like JMS to start a background handler every time someone sends a request ? that way you can control the number of active threads (jms pool size)

Reuse tomcat threads while waiting long time

You need an asynchronous servlet but you also need asynchronous HTTP calls to the external token generator. You will gain nothing by passing the requests from the servlet to an ExecutorService with a thread pool if you still create one thread somewhere per token request. You have to decouple threads from HTTP requests so that one thread can handle multiple HTTP requests. This can be achieved with an asynchronous HTTP client like Apache Asynch HttpClient or Async Http Client.

First you have to create an asynchronous servlet like this one

public class ProxyService extends HttpServlet {

    private CloseableHttpAsyncClient httpClient;

    @Override
    public void init() throws ServletException {
        httpClient = HttpAsyncClients.custom().
                setMaxConnTotal(Integer.parseInt(getInitParameter("maxtotalconnections"))).             
                setMaxConnPerRoute(Integer.parseInt(getInitParameter("maxconnectionsperroute"))).
                build();
        httpClient.start();
    }

    @Override
    public void destroy() {
        try {
            httpClient.close();
        } catch (IOException e) { }
    }

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) {
        AsyncContext asyncCtx = request.startAsync(request, response);
        asyncCtx.setTimeout(ExternalServiceMock.TIMEOUT_SECONDS * ExternalServiceMock.K);       
        ResponseListener listener = new ResponseListener();
        asyncCtx.addListener(listener);
        Future<String> result = httpClient.execute(HttpAsyncMethods.createGet(getInitParameter("serviceurl")), new ResponseConsumer(asyncCtx), null);
    }

}

This servlet performs an asynchronous HTTP call using Apache Asynch HttpClient. Note that you may want to configure the maximum connections per route because as per RFC 2616 spec HttpAsyncClient will only allow a maximum of two concurrent connections to the same host by default. And there are plenty of other options that you can configure as shown in HttpAsyncClient configuration. HttpAsyncClient is expensive to create, therefore you do not want to create an instace of it on each GET operation.

One listener is hooked to the AsyncContext, this listener is only used in the example above to handle timeouts.

public class ResponseListener implements AsyncListener {

    @Override
    public void onStartAsync(AsyncEvent event) throws IOException {
    }

    @Override
    public void onComplete(AsyncEvent event) throws IOException {
    }

    @Override
    public void onError(AsyncEvent event) throws IOException {
        event.getAsyncContext().getResponse().getWriter().print("error:");
    }

    @Override
    public void onTimeout(AsyncEvent event) throws IOException {
        event.getAsyncContext().getResponse().getWriter().print("timeout:");
    }

}

Then you need a consumer for the HTTP client. This consumer informs the AsyncContext by calling complete() when buildResult() is executed internally by HttpClient as a step to return a Future<String> to the caller ProxyService servlet.

public class ResponseConsumer extends AsyncCharConsumer<String> {

    private int responseCode;
    private StringBuilder responseBuffer;
    private AsyncContext asyncCtx;

    public ResponseConsumer(AsyncContext asyncCtx) {
        this.responseBuffer = new StringBuilder();
        this.asyncCtx = asyncCtx;
    }

    @Override
    protected void releaseResources() { }

    @Override
    protected String buildResult(final HttpContext context) {
        try {
            PrintWriter responseWriter = asyncCtx.getResponse().getWriter();
            switch (responseCode) {
                case javax.servlet.http.HttpServletResponse.SC_OK:
                    responseWriter.print("success:" + responseBuffer.toString());
                    break;
                default:
                    responseWriter.print("error:" + responseBuffer.toString());
                }
        } catch (IOException e) { }
        asyncCtx.complete();        
        return responseBuffer.toString();
    }

    @Override
    protected void onCharReceived(CharBuffer buffer, IOControl ioc) throws IOException {
        while (buffer.hasRemaining())
            responseBuffer.append(buffer.get());
    }

    @Override
    protected void onResponseReceived(HttpResponse response) throws HttpException, IOException {        
        responseCode = response.getStatusLine().getStatusCode();
    }

}

The web.xml configuration for ProxyService servlet may be like

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         id="WebApp_ID" version="3.0" metadata-complete="true">
  <display-name>asyncservlet-demo</display-name>

  <servlet>
    <servlet-name>External Service Mock</servlet-name>
    <servlet-class>ExternalServiceMock</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>

  <servlet>
    <servlet-name>Proxy Service</servlet-name>
    <servlet-class>ProxyService</servlet-class>
    <load-on-startup>1</load-on-startup>
    <async-supported>true</async-supported>
    <init-param>
      <param-name>maxtotalconnections</param-name>
      <param-value>200</param-value>
    </init-param>
    <init-param>
      <param-name>maxconnectionsperroute</param-name>
      <param-value>4</param-value>
    </init-param>
    <init-param>
      <param-name>serviceurl</param-name>
      <param-value>http://127.0.0.1:8080/asyncservlet/externalservicemock</param-value>
    </init-param>
  </servlet>

  <servlet-mapping>
    <servlet-name>External Service Mock</servlet-name>
    <url-pattern>/externalservicemock</url-pattern>
  </servlet-mapping>

  <servlet-mapping>
    <servlet-name>Proxy Service</servlet-name>
    <url-pattern>/proxyservice</url-pattern>
  </servlet-mapping>

</web-app>

And a mock servlet for the token generator with a delay in seconds may be:

public class ExternalServiceMock extends HttpServlet{

    public static final int TIMEOUT_SECONDS = 13;
    public static final long K = 1000l;

    public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException {
        Random rnd = new Random();
        try {
            Thread.sleep(rnd.nextInt(TIMEOUT_SECONDS) * K);
        } catch (InterruptedException e) { }
        final byte[] token = String.format("%10d", Math.abs(rnd.nextLong())).getBytes(ISO_8859_1);
        response.setContentType("text/plain");
        response.setCharacterEncoding(ISO_8859_1.name());
        response.setContentLength(token.length);
        response.getOutputStream().write(token);
    }

}

You can get a fully working example at GitHub.

Invoking asynchronous method from servlet

Question 1:

The Servlet Container keeps a pool of threads that it uses to handle requests. It also keeps a single instance of your servlet class. When a request comes in, it dispatches one of those threads which eventually hits the corresponding doXXX method. Threads aren't really queued. They work in parallel. So requests that come in at the same time are being handled parallelly (to a certain degree, read here.

Question 2:

Say you've configured your servlet container to have a pool of 10 threads to handle requests. But at some point you get 15 requests. Then yes, 5 requests will be queued awaiting an available thread. In that case, you can use Servlet 3.0 async support (read1, read2) to have the request handled in a separate thread (that the container also manages). When your processing is complete, a pool thread is re-dispatched to finish the request processing.

Read this for information about how to manage Threads.

What Is Recommended Way for Spawning Threads from a Servlet in Tomcat