{"action":"created","comment":{"url":"https://api.github.com/repos/yahoo/pulsar/pulls/comments/107942321","pull_request_review_id":28945939,"id":107942321,"diff_hunk":"@@ -972,4 +1017,41 @@ public void onUpdate(String path, Map<String, String> data, Stat stat) {\n         return dynamicConfigurationMap;\n     }\n \n-}\n+    /**\n+     * Create pending topic and on completion it picks the next one until processes all topics in\n+     * {@link #pendingTopicLoadingQueue}.<br/>\n+     * It also tries to acquire {@link #topicLoadRequestSemaphore} so throttle down newly incoming topics and release\n+     * permit if it was successful to acquire it.\n+     */\n+    private void createPendingLoadTopic() {\n+        Pair<String, CompletableFuture<Topic>> pendingTopic = pendingTopicLoadingQueue.poll();\n+        if (pendingTopic != null) {\n+            final String topic = pendingTopic.getLeft();\n+            try {\n+                checkTopicNsOwnership(topic);\n+                CompletableFuture<Topic> future = new CompletableFuture<>();","path":"pulsar-broker/src/main/java/com/yahoo/pulsar/broker/service/BrokerService.java","position":161,"original_position":161,"commit_id":"f893c0cd7c861ff07b2f5310d7220cec7f26e1bc","original_commit_id":"f893c0cd7c861ff07b2f5310d7220cec7f26e1bc","user":{"login":"merlimat","id":62500,"avatar_url":"https://avatars1.githubusercontent.com/u/62500?v=3","gravatar_id":"","url":"https://api.github.com/users/merlimat","html_url":"https://github.com/merlimat","followers_url":"https://api.github.com/users/merlimat/followers","following_url":"https://api.github.com/users/merlimat/following{/other_user}","gists_url":"https://api.github.com/users/merlimat/gists{/gist_id}","starred_url":"https://api.github.com/users/merlimat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/merlimat/subscriptions","organizations_url":"https://api.github.com/users/merlimat/orgs","repos_url":"https://api.github.com/users/merlimat/repos","events_url":"https://api.github.com/users/merlimat/events{/privacy}","received_events_url":"https://api.github.com/users/merlimat/received_events","type":"User","site_admin":false},"body":"Same as above, problably we don't need an additional future","created_at":"2017-03-24T16:17:43Z","updated_at":"2017-03-24T16:19:14Z","html_url":"https://github.com/yahoo/pulsar/pull/306#discussion_r107942321","pull_request_url":"https://api.github.com/repos/yahoo/pulsar/pulls/306","_links":{"self":{"href":"https://api.github.com/repos/yahoo/pulsar/pulls/comments/107942321"},"html":{"href":"https://github.com/yahoo/pulsar/pull/306#discussion_r107942321"},"pull_request":{"href":"https://api.github.com/repos/yahoo/pulsar/pulls/306"}}},"pull_request":{"url":"https://api.github.com/repos/yahoo/pulsar/pulls/306","id":112071679,"html_url":"https://github.com/yahoo/pulsar/pull/306","diff_url":"https://github.com/yahoo/pulsar/pull/306.diff","patch_url":"https://github.com/yahoo/pulsar/pull/306.patch","issue_url":"https://api.github.com/repos/yahoo/pulsar/issues/306","number":306,"state":"open","locked":false,"title":"Add topic loading throttling","user":{"login":"rdhabalia","id":2898254,"avatar_url":"https://avatars1.githubusercontent.com/u/2898254?v=3","gravatar_id":"","url":"https://api.github.com/users/rdhabalia","html_url":"https://github.com/rdhabalia","followers_url":"https://api.github.com/users/rdhabalia/followers","following_url":"https://api.github.com/users/rdhabalia/following{/other_user}","gists_url":"https://api.github.com/users/rdhabalia/gists{/gist_id}","starred_url":"https://api.github.com/users/rdhabalia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdhabalia/subscriptions","organizations_url":"https://api.github.com/users/rdhabalia/orgs","repos_url":"https://api.github.com/users/rdhabalia/repos","events_url":"https://api.github.com/users/rdhabalia/events{/privacy}","received_events_url":"https://api.github.com/users/rdhabalia/received_events","type":"User","site_admin":false},"body":"### Motivation\r\n\r\nAddressing #300 : At the time of broker cold-restart, clients make broker to load all the topics concurrently which can create back pressure at zk and may increase zk latency. So, broker should have a way to throttle number of concurrent topic loading.\r\n\r\n### Modifications\r\n\r\n- Introduce dynamic throttling limit to control concurrent topic loading.\r\n- above that limit broker rejects producer/consumer create request so, producer and consumer can do retry with backoff\r\n\r\n### Result\r\n\r\nIt will help broker to control number of zk operations on cold-restart.\r\n","created_at":"2017-03-22T17:50:07Z","updated_at":"2017-03-24T16:19:14Z","closed_at":null,"merged_at":null,"merge_commit_sha":"8f25e528b506e039087c991756ae930e5dab27b5","assignee":{"login":"rdhabalia","id":2898254,"avatar_url":"https://avatars1.githubusercontent.com/u/2898254?v=3","gravatar_id":"","url":"https://api.github.com/users/rdhabalia","html_url":"https://github.com/rdhabalia","followers_url":"https://api.github.com/users/rdhabalia/followers","following_url":"https://api.github.com/users/rdhabalia/following{/other_user}","gists_url":"https://api.github.com/users/rdhabalia/gists{/gist_id}","starred_url":"https://api.github.com/users/rdhabalia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdhabalia/subscriptions","organizations_url":"https://api.github.com/users/rdhabalia/orgs","repos_url":"https://api.github.com/users/rdhabalia/repos","events_url":"https://api.github.com/users/rdhabalia/events{/privacy}","received_events_url":"https://api.github.com/users/rdhabalia/received_events","type":"User","site_admin":false},"assignees":[{"login":"rdhabalia","id":2898254,"avatar_url":"https://avatars1.githubusercontent.com/u/2898254?v=3","gravatar_id":"","url":"https://api.github.com/users/rdhabalia","html_url":"https://github.com/rdhabalia","followers_url":"https://api.github.com/users/rdhabalia/followers","following_url":"https://api.github.com/users/rdhabalia/following{/other_user}","gists_url":"https://api.github.com/users/rdhabalia/gists{/gist_id}","starred_url":"https://api.github.com/users/rdhabalia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdhabalia/subscriptions","organizations_url":"https://api.github.com/users/rdhabalia/orgs","repos_url":"https://api.github.com/users/rdhabalia/repos","events_url":"https://api.github.com/users/rdhabalia/events{/privacy}","received_events_url":"https://api.github.com/users/rdhabalia/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/yahoo/pulsar/milestones/3","html_url":"https://github.com/yahoo/pulsar/milestone/3","labels_url":"https://api.github.com/repos/yahoo/pulsar/milestones/3/labels","id":2090557,"number":3,"title":"1.17","description":"","creator":{"login":"merlimat","id":62500,"avatar_url":"https://avatars1.githubusercontent.com/u/62500?v=3","gravatar_id":"","url":"https://api.github.com/users/merlimat","html_url":"https://github.com/merlimat","followers_url":"https://api.github.com/users/merlimat/followers","following_url":"https://api.github.com/users/merlimat/following{/other_user}","gists_url":"https://api.github.com/users/merlimat/gists{/gist_id}","starred_url":"https://api.github.com/users/merlimat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/merlimat/subscriptions","organizations_url":"https://api.github.com/users/merlimat/orgs","repos_url":"https://api.github.com/users/merlimat/repos","events_url":"https://api.github.com/users/merlimat/events{/privacy}","received_events_url":"https://api.github.com/users/merlimat/received_events","type":"User","site_admin":false},"open_issues":24,"closed_issues":92,"state":"open","created_at":"2016-10-24T20:42:44Z","updated_at":"2017-03-24T06:29:05Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/yahoo/pulsar/pulls/306/commits","review_comments_url":"https://api.github.com/repos/yahoo/pulsar/pulls/306/comments","review_comment_url":"https://api.github.com/repos/yahoo/pulsar/pulls/comments{/number}","comments_url":"https://api.github.com/repos/yahoo/pulsar/issues/306/comments","statuses_url":"https://api.github.com/repos/yahoo/pulsar/statuses/f893c0cd7c861ff07b2f5310d7220cec7f26e1bc","head":{"label":"rdhabalia:topic_throttle","ref":"topic_throttle","sha":"f893c0cd7c861ff07b2f5310d7220cec7f26e1bc","user":{"login":"rdhabalia","id":2898254,"avatar_url":"https://avatars1.githubusercontent.com/u/2898254?v=3","gravatar_id":"","url":"https://api.github.com/users/rdhabalia","html_url":"https://github.com/rdhabalia","followers_url":"https://api.github.com/users/rdhabalia/followers","following_url":"https://api.github.com/users/rdhabalia/following{/other_user}","gists_url":"https://api.github.com/users/rdhabalia/gists{/gist_id}","starred_url":"https://api.github.com/users/rdhabalia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdhabalia/subscriptions","organizations_url":"https://api.github.com/users/rdhabalia/orgs","repos_url":"https://api.github.com/users/rdhabalia/repos","events_url":"https://api.github.com/users/rdhabalia/events{/privacy}","received_events_url":"https://api.github.com/users/rdhabalia/received_events","type":"User","site_admin":false},"repo":{"id":67649596,"name":"pulsar","full_name":"rdhabalia/pulsar","owner":{"login":"rdhabalia","id":2898254,"avatar_url":"https://avatars1.githubusercontent.com/u/2898254?v=3","gravatar_id":"","url":"https://api.github.com/users/rdhabalia","html_url":"https://github.com/rdhabalia","followers_url":"https://api.github.com/users/rdhabalia/followers","following_url":"https://api.github.com/users/rdhabalia/following{/other_user}","gists_url":"https://api.github.com/users/rdhabalia/gists{/gist_id}","starred_url":"https://api.github.com/users/rdhabalia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdhabalia/subscriptions","organizations_url":"https://api.github.com/users/rdhabalia/orgs","repos_url":"https://api.github.com/users/rdhabalia/repos","events_url":"https://api.github.com/users/rdhabalia/events{/privacy}","received_events_url":"https://api.github.com/users/rdhabalia/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/rdhabalia/pulsar","description":"Pulsar - distributed pub-sub messaging system","fork":true,"url":"https://api.github.com/repos/rdhabalia/pulsar","forks_url":"https://api.github.com/repos/rdhabalia/pulsar/forks","keys_url":"https://api.github.com/repos/rdhabalia/pulsar/keys{/key_id}","collaborators_url":"https://api.github.com/repos/rdhabalia/pulsar/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/rdhabalia/pulsar/teams","hooks_url":"https://api.github.com/repos/rdhabalia/pulsar/hooks","issue_events_url":"https://api.github.com/repos/rdhabalia/pulsar/issues/events{/number}","events_url":"https://api.github.com/repos/rdhabalia/pulsar/events","assignees_url":"https://api.github.com/repos/rdhabalia/pulsar/assignees{/user}","branches_url":"https://api.github.com/repos/rdhabalia/pulsar/branches{/branch}","tags_url":"https://api.github.com/repos/rdhabalia/pulsar/tags","blobs_url":"https://api.github.com/repos/rdhabalia/pulsar/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/rdhabalia/pulsar/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/rdhabalia/pulsar/git/refs{/sha}","trees_url":"https://api.github.com/repos/rdhabalia/pulsar/git/trees{/sha}","statuses_url":"https://api.github.com/repos/rdhabalia/pulsar/statuses/{sha}","languages_url":"https://api.github.com/repos/rdhabalia/pulsar/languages","stargazers_url":"https://api.github.com/repos/rdhabalia/pulsar/stargazers","contributors_url":"https://api.github.com/repos/rdhabalia/pulsar/contributors","subscribers_url":"https://api.github.com/repos/rdhabalia/pulsar/subscribers","subscription_url":"https://api.github.com/repos/rdhabalia/pulsar/subscription","commits_url":"https://api.github.com/repos/rdhabalia/pulsar/commits{/sha}","git_commits_url":"https://api.github.com/repos/rdhabalia/pulsar/git/commits{/sha}","comments_url":"https://api.github.com/repos/rdhabalia/pulsar/comments{/number}","issue_comment_url":"https://api.github.com/repos/rdhabalia/pulsar/issues/comments{/number}","contents_url":"https://api.github.com/repos/rdhabalia/pulsar/contents/{+path}","compare_url":"https://api.github.com/repos/rdhabalia/pulsar/compare/{base}...{head}","merges_url":"https://api.github.com/repos/rdhabalia/pulsar/merges","archive_url":"https://api.github.com/repos/rdhabalia/pulsar/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/rdhabalia/pulsar/downloads","issues_url":"https://api.github.com/repos/rdhabalia/pulsar/issues{/number}","pulls_url":"https://api.github.com/repos/rdhabalia/pulsar/pulls{/number}","milestones_url":"https://api.github.com/repos/rdhabalia/pulsar/milestones{/number}","notifications_url":"https://api.github.com/repos/rdhabalia/pulsar/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/rdhabalia/pulsar/labels{/name}","releases_url":"https://api.github.com/repos/rdhabalia/pulsar/releases{/id}","deployments_url":"https://api.github.com/repos/rdhabalia/pulsar/deployments","created_at":"2016-09-07T22:57:02Z","updated_at":"2016-09-07T22:57:05Z","pushed_at":"2017-03-24T08:07:56Z","git_url":"git://github.com/rdhabalia/pulsar.git","ssh_url":"git@github.com:rdhabalia/pulsar.git","clone_url":"https://github.com/rdhabalia/pulsar.git","svn_url":"https://github.com/rdhabalia/pulsar","homepage":"","size":3118,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"yahoo:master","ref":"master","sha":"8cee84677a6d097453e6e80bf3c2b99425bbb3ad","user":{"login":"yahoo","id":16574,"avatar_url":"https://avatars2.githubusercontent.com/u/16574?v=3","gravatar_id":"","url":"https://api.github.com/users/yahoo","html_url":"https://github.com/yahoo","followers_url":"https://api.github.com/users/yahoo/followers","following_url":"https://api.github.com/users/yahoo/following{/other_user}","gists_url":"https://api.github.com/users/yahoo/gists{/gist_id}","starred_url":"https://api.github.com/users/yahoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yahoo/subscriptions","organizations_url":"https://api.github.com/users/yahoo/orgs","repos_url":"https://api.github.com/users/yahoo/repos","events_url":"https://api.github.com/users/yahoo/events{/privacy}","received_events_url":"https://api.github.com/users/yahoo/received_events","type":"Organization","site_admin":false},"repo":{"id":62117812,"name":"pulsar","full_name":"yahoo/pulsar","owner":{"login":"yahoo","id":16574,"avatar_url":"https://avatars2.githubusercontent.com/u/16574?v=3","gravatar_id":"","url":"https://api.github.com/users/yahoo","html_url":"https://github.com/yahoo","followers_url":"https://api.github.com/users/yahoo/followers","following_url":"https://api.github.com/users/yahoo/following{/other_user}","gists_url":"https://api.github.com/users/yahoo/gists{/gist_id}","starred_url":"https://api.github.com/users/yahoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yahoo/subscriptions","organizations_url":"https://api.github.com/users/yahoo/orgs","repos_url":"https://api.github.com/users/yahoo/repos","events_url":"https://api.github.com/users/yahoo/events{/privacy}","received_events_url":"https://api.github.com/users/yahoo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/yahoo/pulsar","description":"Pulsar - distributed pub-sub messaging system","fork":false,"url":"https://api.github.com/repos/yahoo/pulsar","forks_url":"https://api.github.com/repos/yahoo/pulsar/forks","keys_url":"https://api.github.com/repos/yahoo/pulsar/keys{/key_id}","collaborators_url":"https://api.github.com/repos/yahoo/pulsar/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/yahoo/pulsar/teams","hooks_url":"https://api.github.com/repos/yahoo/pulsar/hooks","issue_events_url":"https://api.github.com/repos/yahoo/pulsar/issues/events{/number}","events_url":"https://api.github.com/repos/yahoo/pulsar/events","assignees_url":"https://api.github.com/repos/yahoo/pulsar/assignees{/user}","branches_url":"https://api.github.com/repos/yahoo/pulsar/branches{/branch}","tags_url":"https://api.github.com/repos/yahoo/pulsar/tags","blobs_url":"https://api.github.com/repos/yahoo/pulsar/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/yahoo/pulsar/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/yahoo/pulsar/git/refs{/sha}","trees_url":"https://api.github.com/repos/yahoo/pulsar/git/trees{/sha}","statuses_url":"https://api.github.com/repos/yahoo/pulsar/statuses/{sha}","languages_url":"https://api.github.com/repos/yahoo/pulsar/languages","stargazers_url":"https://api.github.com/repos/yahoo/pulsar/stargazers","contributors_url":"https://api.github.com/repos/yahoo/pulsar/contributors","subscribers_url":"https://api.github.com/repos/yahoo/pulsar/subscribers","subscription_url":"https://api.github.com/repos/yahoo/pulsar/subscription","commits_url":"https://api.github.com/repos/yahoo/pulsar/commits{/sha}","git_commits_url":"https://api.github.com/repos/yahoo/pulsar/git/commits{/sha}","comments_url":"https://api.github.com/repos/yahoo/pulsar/comments{/number}","issue_comment_url":"https://api.github.com/repos/yahoo/pulsar/issues/comments{/number}","contents_url":"https://api.github.com/repos/yahoo/pulsar/contents/{+path}","compare_url":"https://api.github.com/repos/yahoo/pulsar/compare/{base}...{head}","merges_url":"https://api.github.com/repos/yahoo/pulsar/merges","archive_url":"https://api.github.com/repos/yahoo/pulsar/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/yahoo/pulsar/downloads","issues_url":"https://api.github.com/repos/yahoo/pulsar/issues{/number}","pulls_url":"https://api.github.com/repos/yahoo/pulsar/pulls{/number}","milestones_url":"https://api.github.com/repos/yahoo/pulsar/milestones{/number}","notifications_url":"https://api.github.com/repos/yahoo/pulsar/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/yahoo/pulsar/labels{/name}","releases_url":"https://api.github.com/repos/yahoo/pulsar/releases{/id}","deployments_url":"https://api.github.com/repos/yahoo/pulsar/deployments","created_at":"2016-06-28T07:00:03Z","updated_at":"2017-03-23T21:25:57Z","pushed_at":"2017-03-24T08:07:58Z","git_url":"git://github.com/yahoo/pulsar.git","ssh_url":"git@github.com:yahoo/pulsar.git","clone_url":"https://github.com/yahoo/pulsar.git","svn_url":"https://github.com/yahoo/pulsar","homepage":"","size":2958,"stargazers_count":1174,"watchers_count":1174,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":245,"mirror_url":null,"open_issues_count":49,"forks":245,"open_issues":49,"watchers":1174,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/yahoo/pulsar/pulls/306"},"html":{"href":"https://github.com/yahoo/pulsar/pull/306"},"issue":{"href":"https://api.github.com/repos/yahoo/pulsar/issues/306"},"comments":{"href":"https://api.github.com/repos/yahoo/pulsar/issues/306/comments"},"review_comments":{"href":"https://api.github.com/repos/yahoo/pulsar/pulls/306/comments"},"review_comment":{"href":"https://api.github.com/repos/yahoo/pulsar/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/yahoo/pulsar/pulls/306/commits"},"statuses":{"href":"https://api.github.com/repos/yahoo/pulsar/statuses/f893c0cd7c861ff07b2f5310d7220cec7f26e1bc"}}}}
{"actor":{"display_login":"merlimat"}}