title: Apache HttpClient - Out Of Memory Issue
body: <p>I have a processor that reads messages from Apache Kafka and sends the data to a REST Endpoint.</p>
  
  <p>The server only has 4 cores and 4 GB ram, out of which a max of 2GB is allocated to the java process</p>
  
  <p>Messages are produced and consumed at the rate of 4k/second.</p>
  
  <p>After running couple of minutes, the program goes out of memory.</p>
  
  <ul>
  <li>What is the best way to call http rest end-points asynchronously and not wait for response</li>
  <li>How to manage the httpClient connection? I was under the impression that I need to start the client an never close it so I can reuse the connection</li>
  <li>Do you see any issues with the below code</li>
  </ul>
  
  <p>public class SomeProcesor implements BProcessor {</p>
  
  <pre><code>private ThreadPoolExecutor exec = (ThreadPoolExecutor) Executors.newFixedThreadPool(4);
  private CompletionService&lt;Boolean&gt; pool = new ExecutorCompletionService&lt;Boolean&gt;(exec);
  CloseableHttpAsyncClient httpclient = null ; 
  
  @Override
  public void begin() {
      httpclient = HttpAsyncClients.createDefault();
      RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(5000).setConnectTimeout(5000).setSocketTimeout(5000).build();
      HttpAsyncClients.custom().setDefaultRequestConfig(requestConfig).build();
      // Start the client
      httpclient.start();
  
  }
  
  @Override
  public void process(MessageAndMetadata&lt;?, ?&gt; mMData, List&lt;Map&gt; events) {
  
      List&lt;Map&lt;String, Object&gt;&gt; listMap = new ArrayList&lt;&gt;();  
  
      // loop and extract the data from events into the above List
      //..
      //..
  
      // submit to seperate thread to post to HTTP
      pool.submit(new HttpThread(listMap);
  }
  
  private class HttpThread implements Callable&lt;Boolean&gt; {
      List&lt;Map&lt;String, Object&gt;&gt; listMap = null;
      public HttpThread(List&lt;Map&lt;String, Object&gt;&gt; listMap) {
          this.listMap = listMap;
      }
      @Override
      public Boolean call() throws Exception {
          return postToHttp(listMap);
      }
  }
  
  private Boolean postToHttp(List&lt;Map&lt;String, Object&gt;&gt; listMap) throws UnsupportedEncodingException {
      for (Map&lt;String, Object&gt; map : listMap) {
  
          try {
              HttpPost postRequest = new HttpPost("https://myserver:8080/services/collector");
              postRequest.addHeader(HttpHeaders.ACCEPT, "application/json");
              postRequest.addHeader(HttpHeaders.CONTENT_TYPE, "application/json");
              postRequest.addHeader(HttpHeaders.CONNECTION, "keep-alive");
  
              StringEntity input = new StringEntity(methodToConvertMapToJSON(map));
              input.setContentType("application/json");
              postRequest.setEntity(input);
  
              httpclient.execute(postRequest, null);
  
          } catch (Exception e) {
              return false;
          } catch (Throwable th) {
              return false;
          }
      }
      return true;
  }
  </code></pre>
  
  <p>}</p>