{"action":"created","comment":{"url":"https://api.github.com/repos/ceph/ceph/pulls/comments/133099200","pull_request_review_id":56243487,"id":133099200,"diff_hunk":"@@ -633,45 +633,103 @@ int Infiniband::MemoryManager::Cluster::get_buffers(std::vector<Chunk*> &chunks,\n   return r;\n }\n \n-Infiniband::MemoryManager* Infiniband::MemoryManager::RxAllocator::manager = nullptr;\n-PerfCounters *Infiniband::MemoryManager::RxAllocator::perf_logger = nullptr;\n+bool Infiniband::MemoryManager::pool_context::can_alloc(unsigned nbufs)\n+{\n+  /* unimited */\n+  if (manager->cct->_conf->ms_async_rdma_receive_buffers <= 0)\n+    return true;\n+\n+  if (n_bufs_allocated + nbufs > (unsigned)manager->cct->_conf->ms_async_rdma_receive_buffers) {\n+    lderr(manager->cct) << __func__ << \" WARNING: OUT OF RX BUFFERS: allocated: \" <<\n+        n_bufs_allocated << \" requested: \" << nbufs <<\n+        \" limit: \" << manager->cct->_conf->ms_async_rdma_receive_buffers << dendl;\n+    return false;\n+  }\n+\n+  return true;\n+}\n+\n+void Infiniband::MemoryManager::pool_context::set_stat_logger(PerfCounters *logger) {\n+  perf_logger = logger;\n+  if (perf_logger != nullptr)\n+    perf_logger->set(l_msgr_rdma_rx_bufs_total, n_bufs_allocated);\n+}\n \n-unsigned Infiniband::MemoryManager::RxAllocator::n_bufs_allocated = 0;\n-unsigned Infiniband::MemoryManager::RxAllocator::max_bufs         = 0;\n+void Infiniband::MemoryManager::pool_context::update_stats(int nbufs)\n+{\n+  n_bufs_allocated += nbufs;\n+\n+  if (!perf_logger)\n+    return;\n \n+  if (nbufs > 0) {\n+    perf_logger->inc(l_msgr_rdma_rx_bufs_total, nbufs);\n+  } else {\n+    perf_logger->dec(l_msgr_rdma_rx_bufs_total, -nbufs);","path":"src/msg/async/rdma/Infiniband.cc","position":40,"original_position":40,"commit_id":"8fa761f7d4659ef226a21c4b909143ac0f6764b9","original_commit_id":"8fa761f7d4659ef226a21c4b909143ac0f6764b9","user":{"login":"yuyuyu101","id":980577,"avatar_url":"https://avatars2.githubusercontent.com/u/980577?v=4","gravatar_id":"","url":"https://api.github.com/users/yuyuyu101","html_url":"https://github.com/yuyuyu101","followers_url":"https://api.github.com/users/yuyuyu101/followers","following_url":"https://api.github.com/users/yuyuyu101/following{/other_user}","gists_url":"https://api.github.com/users/yuyuyu101/gists{/gist_id}","starred_url":"https://api.github.com/users/yuyuyu101/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yuyuyu101/subscriptions","organizations_url":"https://api.github.com/users/yuyuyu101/orgs","repos_url":"https://api.github.com/users/yuyuyu101/repos","events_url":"https://api.github.com/users/yuyuyu101/events{/privacy}","received_events_url":"https://api.github.com/users/yuyuyu101/received_events","type":"User","site_admin":false},"body":"extra \"-\"","created_at":"2017-08-15T01:12:07Z","updated_at":"2017-08-15T01:17:43Z","html_url":"https://github.com/ceph/ceph/pull/17018#discussion_r133099200","pull_request_url":"https://api.github.com/repos/ceph/ceph/pulls/17018","_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments/133099200"},"html":{"href":"https://github.com/ceph/ceph/pull/17018#discussion_r133099200"},"pull_request":{"href":"https://api.github.com/repos/ceph/ceph/pulls/17018"}}},"pull_request":{"url":"https://api.github.com/repos/ceph/ceph/pulls/17018","id":135608120,"html_url":"https://github.com/ceph/ceph/pull/17018","diff_url":"https://github.com/ceph/ceph/pull/17018.diff","patch_url":"https://github.com/ceph/ceph/pull/17018.patch","issue_url":"https://api.github.com/repos/ceph/ceph/issues/17018","number":17018,"state":"open","locked":false,"title":"msg/async/rdma: refactor rx buffer pool allocator","user":{"login":"alex-mikheev","id":8735389,"avatar_url":"https://avatars3.githubusercontent.com/u/8735389?v=4","gravatar_id":"","url":"https://api.github.com/users/alex-mikheev","html_url":"https://github.com/alex-mikheev","followers_url":"https://api.github.com/users/alex-mikheev/followers","following_url":"https://api.github.com/users/alex-mikheev/following{/other_user}","gists_url":"https://api.github.com/users/alex-mikheev/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-mikheev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-mikheev/subscriptions","organizations_url":"https://api.github.com/users/alex-mikheev/orgs","repos_url":"https://api.github.com/users/alex-mikheev/repos","events_url":"https://api.github.com/users/alex-mikheev/events{/privacy}","received_events_url":"https://api.github.com/users/alex-mikheev/received_events","type":"User","site_admin":false},"body":"@yuyuyu101 please take a look\r\n\r\nPool allocator now has a context. Each context can have a\r\ndifferent configuration. It allows to use buffer pool safely\r\nwhen there are multiple RDMAStack running in parrallel.\r\n\r\nDo not fail on assertion when out of memory for rx buffers.\r\nInstead log warning and try to recover.\r\n\r\nSigned-off-by: Alex Mikheev <alexm@mellanox.com>\r\n\r\nConflicts:\r\n\tsrc/msg/async/rdma/Infiniband.h","created_at":"2017-08-14T14:21:58Z","updated_at":"2017-08-15T01:17:43Z","closed_at":null,"merged_at":null,"merge_commit_sha":"b87c42598ad6cc06ad22e33c3fcba320386cf452","assignee":null,"assignees":[],"requested_reviewers":[],"milestone":null,"commits_url":"https://api.github.com/repos/ceph/ceph/pulls/17018/commits","review_comments_url":"https://api.github.com/repos/ceph/ceph/pulls/17018/comments","review_comment_url":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ceph/ceph/issues/17018/comments","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/8fa761f7d4659ef226a21c4b909143ac0f6764b9","head":{"label":"alex-mikheev:wip_rdma_rxpool_2","ref":"wip_rdma_rxpool_2","sha":"8fa761f7d4659ef226a21c4b909143ac0f6764b9","user":{"login":"alex-mikheev","id":8735389,"avatar_url":"https://avatars3.githubusercontent.com/u/8735389?v=4","gravatar_id":"","url":"https://api.github.com/users/alex-mikheev","html_url":"https://github.com/alex-mikheev","followers_url":"https://api.github.com/users/alex-mikheev/followers","following_url":"https://api.github.com/users/alex-mikheev/following{/other_user}","gists_url":"https://api.github.com/users/alex-mikheev/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-mikheev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-mikheev/subscriptions","organizations_url":"https://api.github.com/users/alex-mikheev/orgs","repos_url":"https://api.github.com/users/alex-mikheev/repos","events_url":"https://api.github.com/users/alex-mikheev/events{/privacy}","received_events_url":"https://api.github.com/users/alex-mikheev/received_events","type":"User","site_admin":false},"repo":{"id":86993056,"name":"ceph","full_name":"alex-mikheev/ceph","owner":{"login":"alex-mikheev","id":8735389,"avatar_url":"https://avatars3.githubusercontent.com/u/8735389?v=4","gravatar_id":"","url":"https://api.github.com/users/alex-mikheev","html_url":"https://github.com/alex-mikheev","followers_url":"https://api.github.com/users/alex-mikheev/followers","following_url":"https://api.github.com/users/alex-mikheev/following{/other_user}","gists_url":"https://api.github.com/users/alex-mikheev/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-mikheev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-mikheev/subscriptions","organizations_url":"https://api.github.com/users/alex-mikheev/orgs","repos_url":"https://api.github.com/users/alex-mikheev/repos","events_url":"https://api.github.com/users/alex-mikheev/events{/privacy}","received_events_url":"https://api.github.com/users/alex-mikheev/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/alex-mikheev/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":true,"url":"https://api.github.com/repos/alex-mikheev/ceph","forks_url":"https://api.github.com/repos/alex-mikheev/ceph/forks","keys_url":"https://api.github.com/repos/alex-mikheev/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/alex-mikheev/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/alex-mikheev/ceph/teams","hooks_url":"https://api.github.com/repos/alex-mikheev/ceph/hooks","issue_events_url":"https://api.github.com/repos/alex-mikheev/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/alex-mikheev/ceph/events","assignees_url":"https://api.github.com/repos/alex-mikheev/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/alex-mikheev/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/alex-mikheev/ceph/tags","blobs_url":"https://api.github.com/repos/alex-mikheev/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/alex-mikheev/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/alex-mikheev/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/alex-mikheev/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/alex-mikheev/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/alex-mikheev/ceph/languages","stargazers_url":"https://api.github.com/repos/alex-mikheev/ceph/stargazers","contributors_url":"https://api.github.com/repos/alex-mikheev/ceph/contributors","subscribers_url":"https://api.github.com/repos/alex-mikheev/ceph/subscribers","subscription_url":"https://api.github.com/repos/alex-mikheev/ceph/subscription","commits_url":"https://api.github.com/repos/alex-mikheev/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/alex-mikheev/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/alex-mikheev/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/alex-mikheev/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/alex-mikheev/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/alex-mikheev/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/alex-mikheev/ceph/merges","archive_url":"https://api.github.com/repos/alex-mikheev/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/alex-mikheev/ceph/downloads","issues_url":"https://api.github.com/repos/alex-mikheev/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/alex-mikheev/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/alex-mikheev/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/alex-mikheev/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/alex-mikheev/ceph/labels{/name}","releases_url":"https://api.github.com/repos/alex-mikheev/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/alex-mikheev/ceph/deployments","created_at":"2017-04-02T14:50:19Z","updated_at":"2017-04-02T14:50:42Z","pushed_at":"2017-08-14T14:09:19Z","git_url":"git://github.com/alex-mikheev/ceph.git","ssh_url":"git@github.com:alex-mikheev/ceph.git","clone_url":"https://github.com/alex-mikheev/ceph.git","svn_url":"https://github.com/alex-mikheev/ceph","homepage":"http://ceph.com","size":231597,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"ceph:master","ref":"master","sha":"c46a964e199876310fd2cebbbcbdbf4c1e4d8d39","user":{"login":"ceph","id":1015767,"avatar_url":"https://avatars3.githubusercontent.com/u/1015767?v=4","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"avatar_url":"https://avatars3.githubusercontent.com/u/1015767?v=4","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/ceph/ceph/deployments","created_at":"2011-09-01T21:41:26Z","updated_at":"2017-08-14T08:48:26Z","pushed_at":"2017-08-15T01:07:18Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":236308,"stargazers_count":3502,"watchers_count":3502,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2040,"mirror_url":null,"open_issues_count":504,"forks":2040,"open_issues":504,"watchers":3502,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/17018"},"html":{"href":"https://github.com/ceph/ceph/pull/17018"},"issue":{"href":"https://api.github.com/repos/ceph/ceph/issues/17018"},"comments":{"href":"https://api.github.com/repos/ceph/ceph/issues/17018/comments"},"review_comments":{"href":"https://api.github.com/repos/ceph/ceph/pulls/17018/comments"},"review_comment":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ceph/ceph/pulls/17018/commits"},"statuses":{"href":"https://api.github.com/repos/ceph/ceph/statuses/8fa761f7d4659ef226a21c4b909143ac0f6764b9"}}}}
{"actor":{"display_login":"yuyuyu101"}}