{"action":"created","issue":{"url":"https://api.github.com/repos/firehol/netdata/issues/2759","repository_url":"https://api.github.com/repos/firehol/netdata","labels_url":"https://api.github.com/repos/firehol/netdata/issues/2759/labels{/name}","comments_url":"https://api.github.com/repos/firehol/netdata/issues/2759/comments","events_url":"https://api.github.com/repos/firehol/netdata/issues/2759/events","html_url":"https://github.com/firehol/netdata/pull/2759","id":258540041,"number":2759,"title":"RFC: Implement nvidia_smi module","user":{"login":"tycho","id":29616,"avatar_url":"https://avatars0.githubusercontent.com/u/29616?v=4","gravatar_id":"","url":"https://api.github.com/users/tycho","html_url":"https://github.com/tycho","followers_url":"https://api.github.com/users/tycho/followers","following_url":"https://api.github.com/users/tycho/following{/other_user}","gists_url":"https://api.github.com/users/tycho/gists{/gist_id}","starred_url":"https://api.github.com/users/tycho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tycho/subscriptions","organizations_url":"https://api.github.com/users/tycho/orgs","repos_url":"https://api.github.com/users/tycho/repos","events_url":"https://api.github.com/users/tycho/events{/privacy}","received_events_url":"https://api.github.com/users/tycho/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":18,"created_at":"2017-09-18T16:21:59Z","updated_at":"2017-09-25T19:24:14Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/firehol/netdata/pulls/2759","html_url":"https://github.com/firehol/netdata/pull/2759","diff_url":"https://github.com/firehol/netdata/pull/2759.diff","patch_url":"https://github.com/firehol/netdata/pull/2759.patch"},"body":"This is mostly ready for merging, I think, but I'd like to do some iterating and get some feedback on it.\r\n\r\nI've been using this on my own machines for a few days without any issues. My dual Xeon box is currently running a TensorFlow learner on the GPU, so you can see this module in action here:\r\n\r\nhttps://netdata.uplinklabs.net/incubus/#menu_gpu_submenu_nvidia_smi;theme=slate;help=true\r\n\r\nA couple of screenshots here, in case it's not busy when you're looking:\r\n\r\n![screenshot_2017-09-18_09-08-26](https://user-images.githubusercontent.com/29616/30551905-001a2350-9c51-11e7-9813-35ed08cb0245.png)\r\n![screenshot_2017-09-18_09-08-20](https://user-images.githubusercontent.com/29616/30551904-00198db4-9c51-11e7-8b1d-9dd99f206e33.png)\r\n\r\nThere are a couple caveats to note. First, I haven't tried this with a multi-GPU box yet, so I'm not sure what the module will do when it sees more than a single GPU. It probably does the wrong thing, but the *intent* is that it creates a separate set of graphs per GPU. \r\n\r\nThere are also some caveats with `nvidia-smi`.\r\n\r\n`nvidia-smi -q` can occasionally be slightly slower when the GPU is under load:\r\n![screenshot_2017-09-18_09-10-19](https://user-images.githubusercontent.com/29616/30551978-46ad2ae2-9c51-11e7-94e7-04e4cf625de2.png)\r\n\r\nAnd it can be **much** slower if the GPU is completely idle, has no X11 session running, and has persistence mode disabled (`nvidia-smi -pm 0`):\r\n![screenshot_2017-09-18_09-13-11](https://user-images.githubusercontent.com/29616/30552070-9e1031a8-9c51-11e7-91ce-fe2b6b063719.png)\r\n"},"comment":{"url":"https://api.github.com/repos/firehol/netdata/issues/comments/331985791","html_url":"https://github.com/firehol/netdata/pull/2759#issuecomment-331985791","issue_url":"https://api.github.com/repos/firehol/netdata/issues/2759","id":331985791,"user":{"login":"ktsaou","id":2662304,"avatar_url":"https://avatars3.githubusercontent.com/u/2662304?v=4","gravatar_id":"","url":"https://api.github.com/users/ktsaou","html_url":"https://github.com/ktsaou","followers_url":"https://api.github.com/users/ktsaou/followers","following_url":"https://api.github.com/users/ktsaou/following{/other_user}","gists_url":"https://api.github.com/users/ktsaou/gists{/gist_id}","starred_url":"https://api.github.com/users/ktsaou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ktsaou/subscriptions","organizations_url":"https://api.github.com/users/ktsaou/orgs","repos_url":"https://api.github.com/users/ktsaou/repos","events_url":"https://api.github.com/users/ktsaou/events{/privacy}","received_events_url":"https://api.github.com/users/ktsaou/received_events","type":"User","site_admin":false},"created_at":"2017-09-25T19:24:14Z","updated_at":"2017-09-25T19:24:14Z","author_association":"OWNER","body":"@l2isbad I can't confirm your observations.\r\n\r\nWith the code in the PR, `python.d.plugin` is runs with 2-3% CPU, while with the new code (`nvidia-smi` always running) it barely consumes 0.5% CPU and `python.d.plugin` runs with 1% CPU.\r\n\r\nI verified `nvidia-smi` is really standing idle between iterations (run it by hand and set `-lms 10000`, and observe it with `top -p $(pidof nvidia-smi)`). \r\n\r\nSo, my conclusion is that if `nvidia-smi` is not always running, `apps.plugin` will report a higher increase for the CPU consumption of `python.d.plugin`.\r\n"}}
{"actor":{"display_login":"ktsaou"}}