title: New to Jetty- waiting for response data with continuations, ASYNCSTARTED exception
body: <p>I'm working on a little proxy server that manages a bunch of tcp-ip connections that run in a selector loop. An external service talks to the proxy server via an http server embedded in the proxy server, and these requests go all the way through to a socket and then return the response from that socket in the response to the original http request.</p>
  
  <p>I have concurrent requests (with a unique request id) coming in on the http server, the actual messages in these requests get stuck on queue and passed along to a bunch of socket connections (a certain socket will correspond to a message), and then the response gets dumped into a hash. The http request checks the hash periodically until it sees a response with the proper request id, or it times out. Here's an awesome system diagram if that makes no sense.</p>
  
  <p><img src="https://i.stack.imgur.com/ScUzS.jpg" alt="Awesome system diagram"></p>
  
  <p>I'm totally new to using Java for http anything, so I figured I'd use Jetty and set up a simple request handler a la the Hello World example. Using a thread.sleep prevents concurrent requests, and continuations as far as I can tell requires events firing. I'd prefer the requests to check the hash once in a while as opposed to firing a connection event each time a socket puts a response in the hash, unless the events would fire by request id or something. That doesn't seem practical if there's a lot of traffic.</p>
  
  <p>Cliffs:</p>
  
  <ul>
  <li>How do I make a handler (or servlet, if I need to go that far) wait
  for a bit or periodically while it's data becomes available without
  blocking everything else?</li>
  <li>Is there a better way to make the http
  request aware of data it should return as a response?</li>
  <li>Is this design bad, and if so, what's the alternative?</li>
  </ul>
  
  <p>EDIT:</p>
  
  <p>Here's my first stab at the problem. It works fine when the HTTP request points at an id which corresponds to a connected tcp client. If the id doesn't correspond to a connected client, jetty goes bonkers and gets into a loop and throws the following exception over and over. What I'd expect to happen is the request to time out because it cannot find any messages in the collector.</p>
  
  <pre><code>Mar 8, 2012 2:39:08 PM APIRequestHandler doPost
  INFO: json: {"state": "255", "m": "set", "id": "00:00:00:00:00:06", "rid": "84955228696f11e1aa04c42c033c8bd7", "sid": "XXXXX"}
  2012-03-08 14:39:08.526:WARN:oejs.ServletHandler:/
  java.lang.IllegalStateException: ASYNCSTARTED,initial
      at org.eclipse.jetty.server.AsyncContinuation.suspend(AsyncContinuation.java:365)
      at org.eclipse.jetty.server.AsyncContinuation.suspend(AsyncContinuation.java:959)
      at APIRequestHandler.doPost(APIRequestHandler.java:72)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
      at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:594)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:485)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:412)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
      at org.eclipse.jetty.server.Server.handle(Server.java:351)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
      at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
      at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)
      at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
      at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
      at java.lang.Thread.run(Thread.java:680)
  </code></pre>
  
  <p>Servlet:</p>
  
  <pre><code>public void doPost(HttpServletRequest req, HttpServletResponse res)
          throws java.io.IOException
      {
      String rid = req.getParameter("rid");
      String msg = req.getParameter("json");
      String id = req.getParameter("id");
  
      // Send to tcp-ip channel
      APIMessage m = new APIMessage();
      m.rid = rid;
      m.id = id;
      m.json = msg + "\r\n";
      queue.add(m);
      this.selector.wakeup();
  
      Continuation cc = ContinuationSupport.getContinuation(req);
  
      res.setContentType("text/plain");
      res.getWriter().println("Request: "+rid+"\tstart:\t"+new Date());
      res.getWriter().flush();
      int elapsed = 0;
      String responseStr = "";
      boolean timeoutHappened = true;
      while(elapsed&lt;this.timeout) // say 50 millis
      {
          if(collector.containsKey(rid)){
              responseStr = collector.get(rid);
              collector.remove(rid);
              timeoutHappened = false;
              break;
          }
          cc.setTimeout(10); // sleep 10 millis
          cc.suspend();
      }
  
      if(!timeoutHappened)
      {
          logger.info("---&gt;API http request found response in collector!");
          logger.info("Respose: "+responseStr);
      }
      else // timed out
      {
          logger.info("---&gt;API http request timed out!");
      }
  
      res.getWriter().println("Request: "+rid+"\tend:\t"+new Date());
      if(cc.isInitial()!=true) {cc.complete();}
  }
  </code></pre>