{"action":"created","comment":{"url":"https://api.github.com/repos/grpc/grpc/pulls/comments/111999660","pull_request_review_id":33280821,"id":111999660,"diff_hunk":"@@ -115,6 +140,39 @@ int grpc_cq_event_timeout_trace;\n static void on_pollset_shutdown_done(grpc_exec_ctx *exec_ctx, void *cc,\n                                      grpc_error *error);\n \n+static void cq_event_queue_init(grpc_cq_event_queue *q) {\n+  gpr_mpscq_init(&q->queue);\n+  gpr_mu_init(&q->queue_mu);\n+  gpr_atm_no_barrier_store(&q->num_queue_items, 0);\n+}\n+\n+static void cq_event_queue_destroy(grpc_cq_event_queue *q) {\n+  gpr_mpscq_destroy(&q->queue);\n+  gpr_mu_destroy(&q->queue_mu);\n+}\n+\n+static void cq_event_queue_push(grpc_cq_event_queue *q, grpc_cq_completion *c) {\n+  gpr_mpscq_push(&q->queue, (gpr_mpscq_node *)c);\n+  gpr_atm_no_barrier_fetch_add(&q->num_queue_items, 1);\n+}\n+\n+static grpc_cq_completion *cq_event_queue_pop(grpc_cq_event_queue *q) {\n+  gpr_mu_lock(&q->queue_mu);","path":"src/core/lib/surface/completion_queue.c","position":77,"original_position":77,"commit_id":"428d707b09743a91459e1c7792117f876ad4597f","original_commit_id":"428d707b09743a91459e1c7792117f876ad4597f","user":{"login":"ctiller","id":10120821,"avatar_url":"https://avatars0.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false},"body":"```c\r\nif (gpr_spinlock_lock(&q->queue_mu)) {\r\n  grpc_cq_completion *c = (grpc_cq_completion *)gpr_mpscq_pop(&q->queue);\r\n  gpr_mu_unlock(&q->queue_mu);\r\n  if (c) {\r\n    gpr_atm_no_barrier_fetch_add(&q->num_queue_items, -1);\r\n  }\r\n  gpr_spinlock_unlock(&q->queue_mu);\r\n  return c;\r\n} else {\r\n  return NULL;\r\n}\r\n```","created_at":"2017-04-18T16:23:36Z","updated_at":"2017-04-18T16:28:49Z","html_url":"https://github.com/grpc/grpc/pull/10662#discussion_r111999660","pull_request_url":"https://api.github.com/repos/grpc/grpc/pulls/10662","_links":{"self":{"href":"https://api.github.com/repos/grpc/grpc/pulls/comments/111999660"},"html":{"href":"https://github.com/grpc/grpc/pull/10662#discussion_r111999660"},"pull_request":{"href":"https://api.github.com/repos/grpc/grpc/pulls/10662"}}},"pull_request":{"url":"https://api.github.com/repos/grpc/grpc/pulls/10662","id":115944841,"html_url":"https://github.com/grpc/grpc/pull/10662","diff_url":"https://github.com/grpc/grpc/pull/10662.diff","patch_url":"https://github.com/grpc/grpc/pull/10662.patch","issue_url":"https://api.github.com/repos/grpc/grpc/issues/10662","number":10662,"state":"open","locked":false,"title":"grpc_mpsc queue based completion queue","user":{"login":"sreecha","id":2754995,"avatar_url":"https://avatars2.githubusercontent.com/u/2754995?v=3","gravatar_id":"","url":"https://api.github.com/users/sreecha","html_url":"https://github.com/sreecha","followers_url":"https://api.github.com/users/sreecha/followers","following_url":"https://api.github.com/users/sreecha/following{/other_user}","gists_url":"https://api.github.com/users/sreecha/gists{/gist_id}","starred_url":"https://api.github.com/users/sreecha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sreecha/subscriptions","organizations_url":"https://api.github.com/users/sreecha/orgs","repos_url":"https://api.github.com/users/sreecha/repos","events_url":"https://api.github.com/users/sreecha/events{/privacy}","received_events_url":"https://api.github.com/users/sreecha/received_events","type":"User","site_admin":false},"body":"Use a lock-free  unbounded multiproducer/singleconsumer queue in `grpc_completion_queue` to store `grpc_cq_completion` objects (instead of using the linked list).\r\n\r\n**`bm_cq_multiple_threads` benchmark results:**\r\n\r\n`bins/opt/bm_cq_multiple_threads --benchmark_min_time=4`\r\n\r\n_Before the change:_\r\n```\r\nBenchmark                                      Time           CPU Iterations\r\n----------------------------------------------------------------------------\r\nBM_Cq_Throughput/real_time/threads:1         173 ns        173 ns   32180442   5.51673M items/s\r\nBM_Cq_Throughput/real_time/threads:2         604 ns       1167 ns    8962716   1.57893M items/s\r\nBM_Cq_Throughput/real_time/threads:4         914 ns       3267 ns    6209900    1068.2k items/s\r\nBM_Cq_Throughput/real_time/threads:8        1103 ns       7089 ns    5045808   885.587k items/s\r\nBM_Cq_Throughput/real_time/threads:16       1091 ns       8306 ns    5166400   895.313k items/s\r\n```\r\n\r\n_After this change:_  \r\nSingle threaded about 30% slower i.e 0.7x  (which can be improved in future by adding a single-thread cq hint) and multiple threads are faster! by 1.3x, 1.5x and 1.9x  (also translates to similar improvements in CPU and wall times)\r\n```\r\n$ bins/opt/bm_cq_multiple_threads --benchmark_min_time=4\r\nBenchmark                                      Time           CPU Iterations\r\n----------------------------------------------------------------------------\r\nBM_Cq_Throughput/real_time/threads:1         241 ns        241 ns   21834512   3.95278M items/s\r\nBM_Cq_Throughput/real_time/threads:2         455 ns        904 ns   16543814   2.09438M items/s\r\nBM_Cq_Throughput/real_time/threads:4         603 ns       2255 ns    9199108   1.58043M items/s\r\nBM_Cq_Throughput/real_time/threads:8         616 ns       3913 ns    8906040   1.54902M items/s\r\nBM_Cq_Throughput/real_time/threads:16        642 ns       5096 ns    9069248   1.48454M items/s\r\n```\r\n","created_at":"2017-04-14T15:15:31Z","updated_at":"2017-04-18T16:28:49Z","closed_at":null,"merged_at":null,"merge_commit_sha":"2bdb4f8bc3023349553319bb3fa43259a0e448a6","assignee":{"login":"ctiller","id":10120821,"avatar_url":"https://avatars0.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false},"assignees":[{"login":"ctiller","id":10120821,"avatar_url":"https://avatars0.githubusercontent.com/u/10120821?v=3","gravatar_id":"","url":"https://api.github.com/users/ctiller","html_url":"https://github.com/ctiller","followers_url":"https://api.github.com/users/ctiller/followers","following_url":"https://api.github.com/users/ctiller/following{/other_user}","gists_url":"https://api.github.com/users/ctiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ctiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ctiller/subscriptions","organizations_url":"https://api.github.com/users/ctiller/orgs","repos_url":"https://api.github.com/users/ctiller/repos","events_url":"https://api.github.com/users/ctiller/events{/privacy}","received_events_url":"https://api.github.com/users/ctiller/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/grpc/grpc/pulls/10662/commits","review_comments_url":"https://api.github.com/repos/grpc/grpc/pulls/10662/comments","review_comment_url":"https://api.github.com/repos/grpc/grpc/pulls/comments{/number}","comments_url":"https://api.github.com/repos/grpc/grpc/issues/10662/comments","statuses_url":"https://api.github.com/repos/grpc/grpc/statuses/428d707b09743a91459e1c7792117f876ad4597f","head":{"label":"sreecha:cq_mpsc_based","ref":"cq_mpsc_based","sha":"428d707b09743a91459e1c7792117f876ad4597f","user":{"login":"sreecha","id":2754995,"avatar_url":"https://avatars2.githubusercontent.com/u/2754995?v=3","gravatar_id":"","url":"https://api.github.com/users/sreecha","html_url":"https://github.com/sreecha","followers_url":"https://api.github.com/users/sreecha/followers","following_url":"https://api.github.com/users/sreecha/following{/other_user}","gists_url":"https://api.github.com/users/sreecha/gists{/gist_id}","starred_url":"https://api.github.com/users/sreecha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sreecha/subscriptions","organizations_url":"https://api.github.com/users/sreecha/orgs","repos_url":"https://api.github.com/users/sreecha/repos","events_url":"https://api.github.com/users/sreecha/events{/privacy}","received_events_url":"https://api.github.com/users/sreecha/received_events","type":"User","site_admin":false},"repo":{"id":43380244,"name":"grpc","full_name":"sreecha/grpc","owner":{"login":"sreecha","id":2754995,"avatar_url":"https://avatars2.githubusercontent.com/u/2754995?v=3","gravatar_id":"","url":"https://api.github.com/users/sreecha","html_url":"https://github.com/sreecha","followers_url":"https://api.github.com/users/sreecha/followers","following_url":"https://api.github.com/users/sreecha/following{/other_user}","gists_url":"https://api.github.com/users/sreecha/gists{/gist_id}","starred_url":"https://api.github.com/users/sreecha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sreecha/subscriptions","organizations_url":"https://api.github.com/users/sreecha/orgs","repos_url":"https://api.github.com/users/sreecha/repos","events_url":"https://api.github.com/users/sreecha/events{/privacy}","received_events_url":"https://api.github.com/users/sreecha/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/sreecha/grpc","description":"The C based gRPC (C++, Node.js, Python, Ruby, Objective-C, PHP, C#)","fork":true,"url":"https://api.github.com/repos/sreecha/grpc","forks_url":"https://api.github.com/repos/sreecha/grpc/forks","keys_url":"https://api.github.com/repos/sreecha/grpc/keys{/key_id}","collaborators_url":"https://api.github.com/repos/sreecha/grpc/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/sreecha/grpc/teams","hooks_url":"https://api.github.com/repos/sreecha/grpc/hooks","issue_events_url":"https://api.github.com/repos/sreecha/grpc/issues/events{/number}","events_url":"https://api.github.com/repos/sreecha/grpc/events","assignees_url":"https://api.github.com/repos/sreecha/grpc/assignees{/user}","branches_url":"https://api.github.com/repos/sreecha/grpc/branches{/branch}","tags_url":"https://api.github.com/repos/sreecha/grpc/tags","blobs_url":"https://api.github.com/repos/sreecha/grpc/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/sreecha/grpc/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/sreecha/grpc/git/refs{/sha}","trees_url":"https://api.github.com/repos/sreecha/grpc/git/trees{/sha}","statuses_url":"https://api.github.com/repos/sreecha/grpc/statuses/{sha}","languages_url":"https://api.github.com/repos/sreecha/grpc/languages","stargazers_url":"https://api.github.com/repos/sreecha/grpc/stargazers","contributors_url":"https://api.github.com/repos/sreecha/grpc/contributors","subscribers_url":"https://api.github.com/repos/sreecha/grpc/subscribers","subscription_url":"https://api.github.com/repos/sreecha/grpc/subscription","commits_url":"https://api.github.com/repos/sreecha/grpc/commits{/sha}","git_commits_url":"https://api.github.com/repos/sreecha/grpc/git/commits{/sha}","comments_url":"https://api.github.com/repos/sreecha/grpc/comments{/number}","issue_comment_url":"https://api.github.com/repos/sreecha/grpc/issues/comments{/number}","contents_url":"https://api.github.com/repos/sreecha/grpc/contents/{+path}","compare_url":"https://api.github.com/repos/sreecha/grpc/compare/{base}...{head}","merges_url":"https://api.github.com/repos/sreecha/grpc/merges","archive_url":"https://api.github.com/repos/sreecha/grpc/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/sreecha/grpc/downloads","issues_url":"https://api.github.com/repos/sreecha/grpc/issues{/number}","pulls_url":"https://api.github.com/repos/sreecha/grpc/pulls{/number}","milestones_url":"https://api.github.com/repos/sreecha/grpc/milestones{/number}","notifications_url":"https://api.github.com/repos/sreecha/grpc/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/sreecha/grpc/labels{/name}","releases_url":"https://api.github.com/repos/sreecha/grpc/releases{/id}","deployments_url":"https://api.github.com/repos/sreecha/grpc/deployments","created_at":"2015-09-29T16:39:17Z","updated_at":"2016-01-11T18:04:02Z","pushed_at":"2017-04-14T16:09:28Z","git_url":"git://github.com/sreecha/grpc.git","ssh_url":"git@github.com:sreecha/grpc.git","clone_url":"https://github.com/sreecha/grpc.git","svn_url":"https://github.com/sreecha/grpc","homepage":"","size":107531,"stargazers_count":0,"watchers_count":0,"language":"C","has_issues":false,"has_projects":true,"has_downloads":false,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":1,"forks":0,"open_issues":1,"watchers":0,"default_branch":"master"}},"base":{"label":"grpc:master","ref":"master","sha":"b81fb794a397b053df0d4bed7b1525a0ff51535f","user":{"login":"grpc","id":7802525,"avatar_url":"https://avatars2.githubusercontent.com/u/7802525?v=3","gravatar_id":"","url":"https://api.github.com/users/grpc","html_url":"https://github.com/grpc","followers_url":"https://api.github.com/users/grpc/followers","following_url":"https://api.github.com/users/grpc/following{/other_user}","gists_url":"https://api.github.com/users/grpc/gists{/gist_id}","starred_url":"https://api.github.com/users/grpc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/grpc/subscriptions","organizations_url":"https://api.github.com/users/grpc/orgs","repos_url":"https://api.github.com/users/grpc/repos","events_url":"https://api.github.com/users/grpc/events{/privacy}","received_events_url":"https://api.github.com/users/grpc/received_events","type":"Organization","site_admin":false},"repo":{"id":27729880,"name":"grpc","full_name":"grpc/grpc","owner":{"login":"grpc","id":7802525,"avatar_url":"https://avatars2.githubusercontent.com/u/7802525?v=3","gravatar_id":"","url":"https://api.github.com/users/grpc","html_url":"https://github.com/grpc","followers_url":"https://api.github.com/users/grpc/followers","following_url":"https://api.github.com/users/grpc/following{/other_user}","gists_url":"https://api.github.com/users/grpc/gists{/gist_id}","starred_url":"https://api.github.com/users/grpc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/grpc/subscriptions","organizations_url":"https://api.github.com/users/grpc/orgs","repos_url":"https://api.github.com/users/grpc/repos","events_url":"https://api.github.com/users/grpc/events{/privacy}","received_events_url":"https://api.github.com/users/grpc/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/grpc/grpc","description":"The C based gRPC (C++, Node.js, Python, Ruby, Objective-C, PHP, C#)","fork":false,"url":"https://api.github.com/repos/grpc/grpc","forks_url":"https://api.github.com/repos/grpc/grpc/forks","keys_url":"https://api.github.com/repos/grpc/grpc/keys{/key_id}","collaborators_url":"https://api.github.com/repos/grpc/grpc/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/grpc/grpc/teams","hooks_url":"https://api.github.com/repos/grpc/grpc/hooks","issue_events_url":"https://api.github.com/repos/grpc/grpc/issues/events{/number}","events_url":"https://api.github.com/repos/grpc/grpc/events","assignees_url":"https://api.github.com/repos/grpc/grpc/assignees{/user}","branches_url":"https://api.github.com/repos/grpc/grpc/branches{/branch}","tags_url":"https://api.github.com/repos/grpc/grpc/tags","blobs_url":"https://api.github.com/repos/grpc/grpc/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/grpc/grpc/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/grpc/grpc/git/refs{/sha}","trees_url":"https://api.github.com/repos/grpc/grpc/git/trees{/sha}","statuses_url":"https://api.github.com/repos/grpc/grpc/statuses/{sha}","languages_url":"https://api.github.com/repos/grpc/grpc/languages","stargazers_url":"https://api.github.com/repos/grpc/grpc/stargazers","contributors_url":"https://api.github.com/repos/grpc/grpc/contributors","subscribers_url":"https://api.github.com/repos/grpc/grpc/subscribers","subscription_url":"https://api.github.com/repos/grpc/grpc/subscription","commits_url":"https://api.github.com/repos/grpc/grpc/commits{/sha}","git_commits_url":"https://api.github.com/repos/grpc/grpc/git/commits{/sha}","comments_url":"https://api.github.com/repos/grpc/grpc/comments{/number}","issue_comment_url":"https://api.github.com/repos/grpc/grpc/issues/comments{/number}","contents_url":"https://api.github.com/repos/grpc/grpc/contents/{+path}","compare_url":"https://api.github.com/repos/grpc/grpc/compare/{base}...{head}","merges_url":"https://api.github.com/repos/grpc/grpc/merges","archive_url":"https://api.github.com/repos/grpc/grpc/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/grpc/grpc/downloads","issues_url":"https://api.github.com/repos/grpc/grpc/issues{/number}","pulls_url":"https://api.github.com/repos/grpc/grpc/pulls{/number}","milestones_url":"https://api.github.com/repos/grpc/grpc/milestones{/number}","notifications_url":"https://api.github.com/repos/grpc/grpc/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/grpc/grpc/labels{/name}","releases_url":"https://api.github.com/repos/grpc/grpc/releases{/id}","deployments_url":"https://api.github.com/repos/grpc/grpc/deployments","created_at":"2014-12-08T18:58:53Z","updated_at":"2017-04-18T12:10:05Z","pushed_at":"2017-04-18T16:19:31Z","git_url":"git://github.com/grpc/grpc.git","ssh_url":"git@github.com:grpc/grpc.git","clone_url":"https://github.com/grpc/grpc.git","svn_url":"https://github.com/grpc/grpc","homepage":"","size":116810,"stargazers_count":9331,"watchers_count":9331,"language":"C","has_issues":true,"has_projects":true,"has_downloads":false,"has_wiki":true,"has_pages":true,"forks_count":1887,"mirror_url":null,"open_issues_count":1180,"forks":1887,"open_issues":1180,"watchers":9331,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/grpc/grpc/pulls/10662"},"html":{"href":"https://github.com/grpc/grpc/pull/10662"},"issue":{"href":"https://api.github.com/repos/grpc/grpc/issues/10662"},"comments":{"href":"https://api.github.com/repos/grpc/grpc/issues/10662/comments"},"review_comments":{"href":"https://api.github.com/repos/grpc/grpc/pulls/10662/comments"},"review_comment":{"href":"https://api.github.com/repos/grpc/grpc/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/grpc/grpc/pulls/10662/commits"},"statuses":{"href":"https://api.github.com/repos/grpc/grpc/statuses/428d707b09743a91459e1c7792117f876ad4597f"}}}}
{"actor":{"display_login":"ctiller"}}