How can I prevent excessive RPC calls by pre-fetching ReferenceProperty properties?
===

I'm trying to show a table of ~800 entities, and having problems keeping it from being really slow.  (Like 15-20 seconds slow.)  I successfully implemented memcache, but because I reference a parent model for each of the child entities it still causes a datastore_v3.Get for each of the 800 and is massively slow.

I then implemented Nick Johnson's [ReferenceProperty prefetching](http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine) and can't solve the following error:

    [... snipped ...]
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2/webapp2.py", line 570, in dispatch
      return method(*args, **kwargs)
    File "/myurl/mypythoncode.py", line 67, in get
      prefetch_refprops(entitylist, ChildModel.parent_program.name)
    File "/myurl/mypythoncode.py", line 36, in prefetch_refprops
      fields = [(entity, prop) for entity in entities for prop in props]
    TypeError: 'NoneType' object is not iterable

**Models:**

These are the two relevant models:

    class ParentModel(db.Model):
      name = db.StringProperty()
      # currently 109 of these

    class ChildModel(db.Model):
      name = db.StringProperty()
      parent_program = db.ReferenceProperty(ParentModel)
      website = db.StringProperty()
      # currently 758 of these

**Python code:**

In my Python code I'm using Nick Johnson's techniques of [efficient model memcaching](http://blog.notdot.net/2009/9/Efficient-model-memcaching) and for [ReferenceProperty prefetching](http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine).  (I've included the ReferenceProperty prefetching below, but not the memcaching code.)

    class AllEntities(webapp2.RequestHandler):
      def get(self):
        entitylist = deserialize_entities(memcache.get("entitylist"))
        entityref = prefetch_refprop(entitylist, ChildModel.parent_program.name)
        if not entitylist:
          entitylist = ChildModel.all().fetch(None)
          entityref = prefetch_refprop(entitylist, ChildModel.parent_program.name)
          memcache.set("entitylist", serialize_entities(entitylist))
        context = {
          'entitylist': entitylist,
        }
        self.response.out.write(template.render(context))

    def prefetch_refprops(entities, *props):
        fields = [(entity, prop) for entity in entities for prop in props]
        ref_keys = [prop.get_value_for_datastore(x) for x, prop in fields]
        ref_entities = dict((x.key(), x) for x in db.get(set(ref_keys)))
        for (entity, prop), ref_key in zip(fields, ref_keys):
            prop.__set__(entity, ref_entities[ref_key])
        return entities

**Jinja2 template:**

My Jinja2 template references the iterable "entry" in "entitylist" but also the parent_program.name and parent_program.key().id()

    {% for entry in entitylist %}
      <tr>
        <td><a href="{{ entry.website}}">{{ entry.website }}</a></td>
        <td><a href="/urlcategory/view?entityid={{ entry.parent_program.key().id() }}">{{ entry.parent_program.name }}</td>
      </tr>
    {% endfor %}


I've replaced the line:

    entityref = prefetch_refprops(entitylist, ChildModel.parent_program.name)

with

    entityref = prefetch_refprops(entitylist, ChildModel.parent_program)

and other variations that include ".name" and ".key().id()".  When I use ".key().id()" I get the error:

    AttributeError: 'ReferenceProperty' object has no attribute 'key'

What am I missing or screwing up?  I'd really appreciate any help!