{"action":"created","comment":{"url":"https://api.github.com/repos/ucbrise/clipper/pulls/comments/123116927","pull_request_review_id":45290063,"id":123116927,"diff_hunk":"@@ -31,13 +32,14 @@ class ModelContainer {\n \n   size_t get_batch_size(Deadline deadline);\n   double get_average_throughput_per_millisecond();\n-  void update_throughput(size_t batch_size, long total_latency);\n+  void update_container_stats(size_t batch_size, long total_latency);\n   void send_feedback(PredictTask task);\n \n   VersionedModelId model_;\n   int container_id_;\n   int replica_id_;\n   InputType input_type_;\n+  clipper::metrics::Histogram latency_hist_;","path":"src/libclipper/include/clipper/containers.hpp","position":20,"original_position":20,"commit_id":"b0032f87851e37dffe322f75f68a8226f6e7e786","original_commit_id":"b0032f87851e37dffe322f75f68a8226f6e7e786","user":{"login":"nishadsingh1","id":6938722,"avatar_url":"https://avatars0.githubusercontent.com/u/6938722?v=3","gravatar_id":"","url":"https://api.github.com/users/nishadsingh1","html_url":"https://github.com/nishadsingh1","followers_url":"https://api.github.com/users/nishadsingh1/followers","following_url":"https://api.github.com/users/nishadsingh1/following{/other_user}","gists_url":"https://api.github.com/users/nishadsingh1/gists{/gist_id}","starred_url":"https://api.github.com/users/nishadsingh1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishadsingh1/subscriptions","organizations_url":"https://api.github.com/users/nishadsingh1/orgs","repos_url":"https://api.github.com/users/nishadsingh1/repos","events_url":"https://api.github.com/users/nishadsingh1/events{/privacy}","received_events_url":"https://api.github.com/users/nishadsingh1/received_events","type":"User","site_admin":false},"body":"This histogram (which receives datapoints here: https://github.com/ucbrise/clipper/pull/225/files#diff-628067215a2c6ded52652fff0b43720bR46) is not used now, but I could see it being involved in some selection policies / task discarding in the future. Happy to remove if this isn't the right time to include it.","created_at":"2017-06-20T22:54:44Z","updated_at":"2017-06-20T22:54:44Z","html_url":"https://github.com/ucbrise/clipper/pull/225#discussion_r123116927","pull_request_url":"https://api.github.com/repos/ucbrise/clipper/pulls/225","_links":{"self":{"href":"https://api.github.com/repos/ucbrise/clipper/pulls/comments/123116927"},"html":{"href":"https://github.com/ucbrise/clipper/pull/225#discussion_r123116927"},"pull_request":{"href":"https://api.github.com/repos/ucbrise/clipper/pulls/225"}}},"pull_request":{"url":"https://api.github.com/repos/ucbrise/clipper/pulls/225","id":126643917,"html_url":"https://github.com/ucbrise/clipper/pull/225","diff_url":"https://github.com/ucbrise/clipper/pull/225.diff","patch_url":"https://github.com/ucbrise/clipper/pull/225.patch","issue_url":"https://api.github.com/repos/ucbrise/clipper/issues/225","number":225,"state":"open","locked":false,"title":"Benchmarking utilities","user":{"login":"nishadsingh1","id":6938722,"avatar_url":"https://avatars0.githubusercontent.com/u/6938722?v=3","gravatar_id":"","url":"https://api.github.com/users/nishadsingh1","html_url":"https://github.com/nishadsingh1","followers_url":"https://api.github.com/users/nishadsingh1/followers","following_url":"https://api.github.com/users/nishadsingh1/following{/other_user}","gists_url":"https://api.github.com/users/nishadsingh1/gists{/gist_id}","starred_url":"https://api.github.com/users/nishadsingh1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishadsingh1/subscriptions","organizations_url":"https://api.github.com/users/nishadsingh1/orgs","repos_url":"https://api.github.com/users/nishadsingh1/repos","events_url":"https://api.github.com/users/nishadsingh1/events{/privacy}","received_events_url":"https://api.github.com/users/nishadsingh1/received_events","type":"User","site_admin":false},"body":"This PR includes some utilities that could be useful for future benchmarking.\r\n\r\nThe changes introduced here are pretty different in nature, so I'm just going to comment on the them directly to give context for why they exist. I'm happy to scrap or rework the ones that are less obviously extensible to other benchmarking use cases.\r\n\r\nThis PR currently does not include the throughput benchmarking script (https://github.com/nishadsingh1/clipper/blob/benchmark/src/benchmarks/src/noop_bench.cpp). Given that we have a benchmarking example in the source, `end_to_end_bench`, and documentation accompanying it at `bench/README.md`, I wonder if it would be good to include a clean version of the throughput benchmaring script too. It could demonstrate usage of some of the benchmarking utilities not used by `end_to_end_bench` which we could also refer to in the documentation. Alternatively, we could move everything under `bench/` and `src/benchmarks/src/` (including the throughput benchmarking script) to some other branch with updated documentation at `bench/README.md`.\r\n","created_at":"2017-06-20T22:29:39Z","updated_at":"2017-06-20T22:54:44Z","closed_at":null,"merged_at":null,"merge_commit_sha":"afbde872abaeafb0846eb0a720327794d590c319","assignee":{"login":"nishadsingh1","id":6938722,"avatar_url":"https://avatars0.githubusercontent.com/u/6938722?v=3","gravatar_id":"","url":"https://api.github.com/users/nishadsingh1","html_url":"https://github.com/nishadsingh1","followers_url":"https://api.github.com/users/nishadsingh1/followers","following_url":"https://api.github.com/users/nishadsingh1/following{/other_user}","gists_url":"https://api.github.com/users/nishadsingh1/gists{/gist_id}","starred_url":"https://api.github.com/users/nishadsingh1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishadsingh1/subscriptions","organizations_url":"https://api.github.com/users/nishadsingh1/orgs","repos_url":"https://api.github.com/users/nishadsingh1/repos","events_url":"https://api.github.com/users/nishadsingh1/events{/privacy}","received_events_url":"https://api.github.com/users/nishadsingh1/received_events","type":"User","site_admin":false},"assignees":[{"login":"nishadsingh1","id":6938722,"avatar_url":"https://avatars0.githubusercontent.com/u/6938722?v=3","gravatar_id":"","url":"https://api.github.com/users/nishadsingh1","html_url":"https://github.com/nishadsingh1","followers_url":"https://api.github.com/users/nishadsingh1/followers","following_url":"https://api.github.com/users/nishadsingh1/following{/other_user}","gists_url":"https://api.github.com/users/nishadsingh1/gists{/gist_id}","starred_url":"https://api.github.com/users/nishadsingh1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishadsingh1/subscriptions","organizations_url":"https://api.github.com/users/nishadsingh1/orgs","repos_url":"https://api.github.com/users/nishadsingh1/repos","events_url":"https://api.github.com/users/nishadsingh1/events{/privacy}","received_events_url":"https://api.github.com/users/nishadsingh1/received_events","type":"User","site_admin":false}],"requested_reviewers":[{"login":"dcrankshaw","id":431588,"avatar_url":"https://avatars2.githubusercontent.com/u/431588?v=3","gravatar_id":"","url":"https://api.github.com/users/dcrankshaw","html_url":"https://github.com/dcrankshaw","followers_url":"https://api.github.com/users/dcrankshaw/followers","following_url":"https://api.github.com/users/dcrankshaw/following{/other_user}","gists_url":"https://api.github.com/users/dcrankshaw/gists{/gist_id}","starred_url":"https://api.github.com/users/dcrankshaw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dcrankshaw/subscriptions","organizations_url":"https://api.github.com/users/dcrankshaw/orgs","repos_url":"https://api.github.com/users/dcrankshaw/repos","events_url":"https://api.github.com/users/dcrankshaw/events{/privacy}","received_events_url":"https://api.github.com/users/dcrankshaw/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/ucbrise/clipper/pulls/225/commits","review_comments_url":"https://api.github.com/repos/ucbrise/clipper/pulls/225/comments","review_comment_url":"https://api.github.com/repos/ucbrise/clipper/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ucbrise/clipper/issues/225/comments","statuses_url":"https://api.github.com/repos/ucbrise/clipper/statuses/b0032f87851e37dffe322f75f68a8226f6e7e786","head":{"label":"nishadsingh1:benchmark_infra","ref":"benchmark_infra","sha":"b0032f87851e37dffe322f75f68a8226f6e7e786","user":{"login":"nishadsingh1","id":6938722,"avatar_url":"https://avatars0.githubusercontent.com/u/6938722?v=3","gravatar_id":"","url":"https://api.github.com/users/nishadsingh1","html_url":"https://github.com/nishadsingh1","followers_url":"https://api.github.com/users/nishadsingh1/followers","following_url":"https://api.github.com/users/nishadsingh1/following{/other_user}","gists_url":"https://api.github.com/users/nishadsingh1/gists{/gist_id}","starred_url":"https://api.github.com/users/nishadsingh1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishadsingh1/subscriptions","organizations_url":"https://api.github.com/users/nishadsingh1/orgs","repos_url":"https://api.github.com/users/nishadsingh1/repos","events_url":"https://api.github.com/users/nishadsingh1/events{/privacy}","received_events_url":"https://api.github.com/users/nishadsingh1/received_events","type":"User","site_admin":false},"repo":{"id":84875757,"name":"clipper","full_name":"nishadsingh1/clipper","owner":{"login":"nishadsingh1","id":6938722,"avatar_url":"https://avatars0.githubusercontent.com/u/6938722?v=3","gravatar_id":"","url":"https://api.github.com/users/nishadsingh1","html_url":"https://github.com/nishadsingh1","followers_url":"https://api.github.com/users/nishadsingh1/followers","following_url":"https://api.github.com/users/nishadsingh1/following{/other_user}","gists_url":"https://api.github.com/users/nishadsingh1/gists{/gist_id}","starred_url":"https://api.github.com/users/nishadsingh1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nishadsingh1/subscriptions","organizations_url":"https://api.github.com/users/nishadsingh1/orgs","repos_url":"https://api.github.com/users/nishadsingh1/repos","events_url":"https://api.github.com/users/nishadsingh1/events{/privacy}","received_events_url":"https://api.github.com/users/nishadsingh1/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/nishadsingh1/clipper","description":"A low-latency prediction-serving system","fork":true,"url":"https://api.github.com/repos/nishadsingh1/clipper","forks_url":"https://api.github.com/repos/nishadsingh1/clipper/forks","keys_url":"https://api.github.com/repos/nishadsingh1/clipper/keys{/key_id}","collaborators_url":"https://api.github.com/repos/nishadsingh1/clipper/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/nishadsingh1/clipper/teams","hooks_url":"https://api.github.com/repos/nishadsingh1/clipper/hooks","issue_events_url":"https://api.github.com/repos/nishadsingh1/clipper/issues/events{/number}","events_url":"https://api.github.com/repos/nishadsingh1/clipper/events","assignees_url":"https://api.github.com/repos/nishadsingh1/clipper/assignees{/user}","branches_url":"https://api.github.com/repos/nishadsingh1/clipper/branches{/branch}","tags_url":"https://api.github.com/repos/nishadsingh1/clipper/tags","blobs_url":"https://api.github.com/repos/nishadsingh1/clipper/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/nishadsingh1/clipper/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/nishadsingh1/clipper/git/refs{/sha}","trees_url":"https://api.github.com/repos/nishadsingh1/clipper/git/trees{/sha}","statuses_url":"https://api.github.com/repos/nishadsingh1/clipper/statuses/{sha}","languages_url":"https://api.github.com/repos/nishadsingh1/clipper/languages","stargazers_url":"https://api.github.com/repos/nishadsingh1/clipper/stargazers","contributors_url":"https://api.github.com/repos/nishadsingh1/clipper/contributors","subscribers_url":"https://api.github.com/repos/nishadsingh1/clipper/subscribers","subscription_url":"https://api.github.com/repos/nishadsingh1/clipper/subscription","commits_url":"https://api.github.com/repos/nishadsingh1/clipper/commits{/sha}","git_commits_url":"https://api.github.com/repos/nishadsingh1/clipper/git/commits{/sha}","comments_url":"https://api.github.com/repos/nishadsingh1/clipper/comments{/number}","issue_comment_url":"https://api.github.com/repos/nishadsingh1/clipper/issues/comments{/number}","contents_url":"https://api.github.com/repos/nishadsingh1/clipper/contents/{+path}","compare_url":"https://api.github.com/repos/nishadsingh1/clipper/compare/{base}...{head}","merges_url":"https://api.github.com/repos/nishadsingh1/clipper/merges","archive_url":"https://api.github.com/repos/nishadsingh1/clipper/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/nishadsingh1/clipper/downloads","issues_url":"https://api.github.com/repos/nishadsingh1/clipper/issues{/number}","pulls_url":"https://api.github.com/repos/nishadsingh1/clipper/pulls{/number}","milestones_url":"https://api.github.com/repos/nishadsingh1/clipper/milestones{/number}","notifications_url":"https://api.github.com/repos/nishadsingh1/clipper/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/nishadsingh1/clipper/labels{/name}","releases_url":"https://api.github.com/repos/nishadsingh1/clipper/releases{/id}","deployments_url":"https://api.github.com/repos/nishadsingh1/clipper/deployments","created_at":"2017-03-13T21:20:23Z","updated_at":"2017-03-13T21:20:25Z","pushed_at":"2017-06-20T22:14:02Z","git_url":"git://github.com/nishadsingh1/clipper.git","ssh_url":"git@github.com:nishadsingh1/clipper.git","clone_url":"https://github.com/nishadsingh1/clipper.git","svn_url":"https://github.com/nishadsingh1/clipper","homepage":"","size":22605,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"develop"}},"base":{"label":"ucbrise:develop","ref":"develop","sha":"e71d594f9439620dd937e07ef92fd3c376a757cc","user":{"login":"ucbrise","id":20959024,"avatar_url":"https://avatars1.githubusercontent.com/u/20959024?v=3","gravatar_id":"","url":"https://api.github.com/users/ucbrise","html_url":"https://github.com/ucbrise","followers_url":"https://api.github.com/users/ucbrise/followers","following_url":"https://api.github.com/users/ucbrise/following{/other_user}","gists_url":"https://api.github.com/users/ucbrise/gists{/gist_id}","starred_url":"https://api.github.com/users/ucbrise/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ucbrise/subscriptions","organizations_url":"https://api.github.com/users/ucbrise/orgs","repos_url":"https://api.github.com/users/ucbrise/repos","events_url":"https://api.github.com/users/ucbrise/events{/privacy}","received_events_url":"https://api.github.com/users/ucbrise/received_events","type":"Organization","site_admin":false},"repo":{"id":72132009,"name":"clipper","full_name":"ucbrise/clipper","owner":{"login":"ucbrise","id":20959024,"avatar_url":"https://avatars1.githubusercontent.com/u/20959024?v=3","gravatar_id":"","url":"https://api.github.com/users/ucbrise","html_url":"https://github.com/ucbrise","followers_url":"https://api.github.com/users/ucbrise/followers","following_url":"https://api.github.com/users/ucbrise/following{/other_user}","gists_url":"https://api.github.com/users/ucbrise/gists{/gist_id}","starred_url":"https://api.github.com/users/ucbrise/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ucbrise/subscriptions","organizations_url":"https://api.github.com/users/ucbrise/orgs","repos_url":"https://api.github.com/users/ucbrise/repos","events_url":"https://api.github.com/users/ucbrise/events{/privacy}","received_events_url":"https://api.github.com/users/ucbrise/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ucbrise/clipper","description":"A low-latency prediction-serving system","fork":false,"url":"https://api.github.com/repos/ucbrise/clipper","forks_url":"https://api.github.com/repos/ucbrise/clipper/forks","keys_url":"https://api.github.com/repos/ucbrise/clipper/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ucbrise/clipper/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ucbrise/clipper/teams","hooks_url":"https://api.github.com/repos/ucbrise/clipper/hooks","issue_events_url":"https://api.github.com/repos/ucbrise/clipper/issues/events{/number}","events_url":"https://api.github.com/repos/ucbrise/clipper/events","assignees_url":"https://api.github.com/repos/ucbrise/clipper/assignees{/user}","branches_url":"https://api.github.com/repos/ucbrise/clipper/branches{/branch}","tags_url":"https://api.github.com/repos/ucbrise/clipper/tags","blobs_url":"https://api.github.com/repos/ucbrise/clipper/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ucbrise/clipper/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ucbrise/clipper/git/refs{/sha}","trees_url":"https://api.github.com/repos/ucbrise/clipper/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ucbrise/clipper/statuses/{sha}","languages_url":"https://api.github.com/repos/ucbrise/clipper/languages","stargazers_url":"https://api.github.com/repos/ucbrise/clipper/stargazers","contributors_url":"https://api.github.com/repos/ucbrise/clipper/contributors","subscribers_url":"https://api.github.com/repos/ucbrise/clipper/subscribers","subscription_url":"https://api.github.com/repos/ucbrise/clipper/subscription","commits_url":"https://api.github.com/repos/ucbrise/clipper/commits{/sha}","git_commits_url":"https://api.github.com/repos/ucbrise/clipper/git/commits{/sha}","comments_url":"https://api.github.com/repos/ucbrise/clipper/comments{/number}","issue_comment_url":"https://api.github.com/repos/ucbrise/clipper/issues/comments{/number}","contents_url":"https://api.github.com/repos/ucbrise/clipper/contents/{+path}","compare_url":"https://api.github.com/repos/ucbrise/clipper/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ucbrise/clipper/merges","archive_url":"https://api.github.com/repos/ucbrise/clipper/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ucbrise/clipper/downloads","issues_url":"https://api.github.com/repos/ucbrise/clipper/issues{/number}","pulls_url":"https://api.github.com/repos/ucbrise/clipper/pulls{/number}","milestones_url":"https://api.github.com/repos/ucbrise/clipper/milestones{/number}","notifications_url":"https://api.github.com/repos/ucbrise/clipper/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ucbrise/clipper/labels{/name}","releases_url":"https://api.github.com/repos/ucbrise/clipper/releases{/id}","deployments_url":"https://api.github.com/repos/ucbrise/clipper/deployments","created_at":"2016-10-27T17:30:27Z","updated_at":"2017-06-20T22:34:39Z","pushed_at":"2017-06-20T22:29:39Z","git_url":"git://github.com/ucbrise/clipper.git","ssh_url":"git@github.com:ucbrise/clipper.git","clone_url":"https://github.com/ucbrise/clipper.git","svn_url":"https://github.com/ucbrise/clipper","homepage":"http://clipper.ai","size":22123,"stargazers_count":168,"watchers_count":168,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":39,"mirror_url":null,"open_issues_count":25,"forks":39,"open_issues":25,"watchers":168,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/ucbrise/clipper/pulls/225"},"html":{"href":"https://github.com/ucbrise/clipper/pull/225"},"issue":{"href":"https://api.github.com/repos/ucbrise/clipper/issues/225"},"comments":{"href":"https://api.github.com/repos/ucbrise/clipper/issues/225/comments"},"review_comments":{"href":"https://api.github.com/repos/ucbrise/clipper/pulls/225/comments"},"review_comment":{"href":"https://api.github.com/repos/ucbrise/clipper/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ucbrise/clipper/pulls/225/commits"},"statuses":{"href":"https://api.github.com/repos/ucbrise/clipper/statuses/b0032f87851e37dffe322f75f68a8226f6e7e786"}}}}
{"actor":{"display_login":"nishadsingh1"}}