title: Sorting CouchDB data using couch-lucene
body: <p>I have some summation data that was very easy to generate using some relatively simple map/reduce views.  But we want to sort the data based on the group-reduced view <em>values</em> (not the keys).  It was suggested that we could use <a href="http://github.com/rnewson/couchdb-lucene/" rel="nofollow noreferrer">couchdb-lucene</a> to do this.  But how?  It's not clear to me how to use a full text index to quickly rank this sort of data.</p>
  
  <p><strong>What we already have</strong></p>
  
  <p>An oversimplified example view looks something like the following:</p>
  
  <pre><code>by_sender: {
    map: "function(doc) { emit(doc.sender, 1); }",
    reduce: "function(keys, values, rereduce) { return sum(values); }"
  }
  </code></pre>
  
  <p>Which returns results somewhat like the following (when run with <code>group=true</code>):</p>
  
  <pre><code> {"rows":[
   {"key":"a@example.com","value":2},
   {"key":"aaa@example.com","value":1},
   {"key":"aaap@example.com","value":34},
   {"key":"aabb@example.com","value":1},
   ... thousands or tens of thousands of rows ...
   ]}
  </code></pre>
  
  <p><strong>What we want</strong></p>
  
  <p>Those are sorted by the key, but I need to sort it data according the values, like so:</p>
  
  <pre><code> {"rows":[
   {"key":"xyzzy@example.com","value":847},
   {"key":"adam@example.com","value":345},
   {"key":"karl@example.com","value":99},
   {"key":"aaap@example.com","value":34},
   ... thousands or tens of thousands of rows ...
   ]}
  </code></pre>
  
  <p>And I need it sorted as quickly as is reasonably possible (e.g. if it only takes &lt;100ms to update the indexes, it shouldn't take 1 minute before the new data is reflected in queries).</p>
  
  <p><strong>More context: what we already tried</strong></p>
  
  <p>The best answer on <a href="https://stackoverflow.com/questions/2817703/sorting-couchdb-views-by-value">Sorting CouchDB Views By Value</a> gives four viable options, which we've tried in increasing order of difficulty:</p>
  
  <ol>
  <li>First we sorted the results client side, but that was <em>way</em> too slow.</li>
  <li>Next we created a list function which sorts the data.  A little faster, but still too slow.</li>
  <li>Chained Map-Reduce Views should handle this problem easily.
  
  <ul>
  <li>Someone pointed out Cloudant's <a href="http://support.cloudant.com/customer/portal/articles/359310-chained-mapreduce-views" rel="nofollow noreferrer">Chained Map-Reduce Views</a>.  They are not in  <a href="http://bigcouch.cloudant.com/" rel="nofollow noreferrer">BigCouch</a> but are part of Cloudant's services, which are unfortunately not in our budget at this time. </li>
  <li>I started an application layer implementation using the _bulk_docs API.  It is tricky if you want to keep updates as snappy as possible while avoiding race conditions, etc.  I can continue with this approach, but it is <em>not</em> relaxing.  :(</li>
  </ul></li>
  <li>The answer suggested using <a href="http://github.com/rnewson/couchdb-lucene/" rel="nofollow noreferrer">couchdb-lucene</a>.  But I'm not nearly familiar enough with full-text search to understand how to get it to do anything more sophisticated than index the document and return a search result.  I don't even know where to start.</li>
  </ol>