title: MongoDB's performance on aggregation queries
body: <p>After hearing so many good things about MongoDB's performance we decided to give Mongodb a try to solve a problem we have. I started by moving all the records we have in several mysql databases to a single collection in mongodb. This resulted in a collection with <strong>29 Million documents</strong> (each one of them have at least 20 fields) which takes around 100 GB of space in the HD. We decided to put them all in one collection since all the documents have the same structure and we want to query and aggregate results on all those documents. </p>
  
  <p>I created some indexes to match my queries otherwise even a simple count() would take ages. However, queries such as distinct() and group() still take way too long. </p>
  
  <p>Example:</p>
  
  <pre><code>// creation of a compound index    
  db.collection.ensureIndex({'metadata.system':1, 'metadata.company':1})
  
  // query to get all the combinations companies and systems
  db.collection.group({key: { 'metadata.system':true, 'metadata.company':true }, reduce: function(obj,prev) {}, initial: {} });
  </code></pre>
  
  <p>I took a look at the mongod log and it has a lot of lines like these (while executing the query above):</p>
  
  <pre><code>Thu Apr  8 14:40:05 getmore database.collection cid:973023491046432059 ntoreturn:0 query: {}  bytes:1048890 nreturned:417 154ms
  Thu Apr  8 14:40:08 getmore database.collection cid:973023491046432059 ntoreturn:0 query: {}  bytes:1050205 nreturned:414 430ms
  Thu Apr  8 14:40:18 getmore database.collection cid:973023491046432059 ntoreturn:0 query: {}  bytes:1049748 nreturned:201 130ms
  Thu Apr  8 14:40:27 getmore database.collection cid:973023491046432059 ntoreturn:0 query: {}  bytes:1051925 nreturned:221 118ms
  Thu Apr  8 14:40:30 getmore database.collection cid:973023491046432059 ntoreturn:0 query: {}  bytes:1053096 nreturned:250 164ms
  ...
  Thu Apr  8 15:04:18 query database.$cmd ntoreturn:1 command  reslen:4130 1475894ms
  </code></pre>
  
  <p>This query took 1475894ms which is way longer than what I would expect (the result list has around 60 entries). First of all, is this expected given the large number of documents in my collection? Are aggregation queries in general expected to be so slow in mongodb? Any thoughts on how can I improve the performance? </p>
  
  <p>I am running mongod in a single machine with a dual core and 10GB of memory. </p>
  
  <p>Thank you.</p>