{"action":"created","issue":{"url":"https://api.github.com/repos/takluyver/entrypoints/issues/16","repository_url":"https://api.github.com/repos/takluyver/entrypoints","labels_url":"https://api.github.com/repos/takluyver/entrypoints/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/takluyver/entrypoints/issues/16/comments","events_url":"https://api.github.com/repos/takluyver/entrypoints/issues/16/events","html_url":"https://github.com/takluyver/entrypoints/issues/16","id":278202752,"number":16,"title":"Caching entry point data","user":{"login":"takluyver","id":327925,"avatar_url":"https://avatars3.githubusercontent.com/u/327925?v=4","gravatar_id":"","url":"https://api.github.com/users/takluyver","html_url":"https://github.com/takluyver","followers_url":"https://api.github.com/users/takluyver/followers","following_url":"https://api.github.com/users/takluyver/following{/other_user}","gists_url":"https://api.github.com/users/takluyver/gists{/gist_id}","starred_url":"https://api.github.com/users/takluyver/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/takluyver/subscriptions","organizations_url":"https://api.github.com/users/takluyver/orgs","repos_url":"https://api.github.com/users/takluyver/repos","events_url":"https://api.github.com/users/takluyver/events{/privacy}","received_events_url":"https://api.github.com/users/takluyver/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-11-30T17:21:34Z","updated_at":"2017-11-30T17:44:31Z","closed_at":null,"author_association":"OWNER","body":"This issue is to discuss how best to cache entry points, and how to ensure that it's performant and accurate."},"comment":{"url":"https://api.github.com/repos/takluyver/entrypoints/issues/comments/348265133","html_url":"https://github.com/takluyver/entrypoints/issues/16#issuecomment-348265133","issue_url":"https://api.github.com/repos/takluyver/entrypoints/issues/16","id":348265133,"user":{"login":"takluyver","id":327925,"avatar_url":"https://avatars3.githubusercontent.com/u/327925?v=4","gravatar_id":"","url":"https://api.github.com/users/takluyver","html_url":"https://github.com/takluyver","followers_url":"https://api.github.com/users/takluyver/followers","following_url":"https://api.github.com/users/takluyver/following{/other_user}","gists_url":"https://api.github.com/users/takluyver/gists{/gist_id}","starred_url":"https://api.github.com/users/takluyver/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/takluyver/subscriptions","organizations_url":"https://api.github.com/users/takluyver/orgs","repos_url":"https://api.github.com/users/takluyver/repos","events_url":"https://api.github.com/users/takluyver/events{/privacy}","received_events_url":"https://api.github.com/users/takluyver/received_events","type":"User","site_admin":false},"created_at":"2017-11-30T17:44:31Z","updated_at":"2017-11-30T17:44:31Z","author_association":"OWNER","body":"I have been experimenting with caching on the [cache-experimentation branch](https://github.com/takluyver/entrypoints/tree/cache-experimentation). I tested one implementation with a cache file stored in each `sys.path` directory (`make_cache.py` and `read_cache.py`) and one with a single cache file with directories as keys (`*_single_cache.py`).\r\n\r\n**Cold start performance** - where the files it reads are not cached in memory by the OS - tested using a command from [here](https://wiki.openoffice.org/wiki/Cold-start-simulator), on a system with a spinning hard drive (not an SSD). This is a worst case for this kind of operation, but it's realistic.\r\n\r\n- Loading entrypoints with no cache files: 4.5 seconds\r\n- Loading with a cache per directory: 0.44 seconds\r\n- Loading with a single cache: 0.40 seconds\r\n- Running Python with a no-op file: 0.05 seconds\r\n\r\nHowever, my system has a short `sys.path`, with only 7 directories on it. If you have more a longer sys.path, e.g. because you're using 'egg' installations, the difference between per-directory caches and a single cache will be bigger.\r\n\r\n**Warm start performance** - with the files in memory - didn't change much, from about 110ms with no cache to about 85ms with a single cache.\r\n\r\n-----\r\n\r\nFull timing results (sorry, it's handwritten):\r\n\r\n![spr_176_r5017113016590](https://user-images.githubusercontent.com/327925/33445844-123128de-d5f6-11e7-9ebd-a77b468f935b.jpg)\r\n\r\n"}}
{"actor":{"display_login":"takluyver"}}