title: Adapter Proxy for Restful APIs
body: <p>this is a general 'what technologies are available' question.</p>
  
  <p>My company provides a web application with a RESTful API.  However, it is too slow for my needs and some of the results are in an awkward format.</p>
  
  <p><img src="https://i.stack.imgur.com/qpLRf.png" alt="enter image description here"></p>
  
  <p>I want to wrap their restful server with a proxy/adapter server, so when you connect to the proxy you get the RESTful API I <em>wish</em> the real one provides.</p>
  
  <p><img src="https://i.stack.imgur.com/1R6T1.png" alt="enter image description here"></p>
  
  <p>So it needs to do a few things:</p>
  
  <ul>
  <li>passthrough most requests</li>
  <li>cache some requests</li>
  <li>do some extra requests on the original server to detect if a request is cacheable</li>
  </ul>
  
  <p><img src="https://i.stack.imgur.com/3YK4b.png" alt="enter image description here"></p>
  
  <p>for instance: there is a request for a field in a record: GET /records/id/field which might be slow, but there is a fingerprint request GET /records/id/fingerprint which is always fast.  If there exists a cache of GET /records/1/field2 for the fingerprint <code>feedbeef</code>, then I need to check the original server still has the fingerprint <code>feed beef</code> before serving the cached version.  </p>
  
  <ul>
  <li>fix headers for some responses - e.g. content-type, based upon the path</li>
  <li>do stream processing on some large content, for instance</li>
  </ul>
  
  <p>GET /records/id/attachments/1234</p>
  
  <p><img src="https://i.stack.imgur.com/O20nH.png" alt="enter image description here"></p>
  
  <p>returns a 100Mb log file in text format</p>
  
  <p>remove null characters from files
  optionally recode the log to filter out irrelevant lines, reducing the load on the client
  cache the filtered version for later requests.</p>
  
  <hr>
  
  <p>While I could modify the client to achieve this functionality, such code would not be re-usable for other clients (different languages), and complicates the client logic.</p>
  
  <hr>
  
  <p>I had a look at whether clojure/ring could do it, and while there is a nice little proxy middleware for it, it doesn't handle streaming content as far as I can tell - the whole 100Mb would have to be downloaded.  Also it doesn't include any cache logic yet.</p>
  
  <hr>
  
  <p>I took a look at whether squid could do it, but I'm not familiar with the technology, and it seems mostly concerned with passing through requests rather than modifying them on the fly.</p>
  
  <hr>
  
  <p>I'm looking for hints where I might find the correct technology to implement this.  I'm mostly language agnostic if learning a new language gets me access to a really simple way to do it.</p>