{"action":"created","issue":{"url":"https://api.github.com/repos/hashicorp/nomad/issues/3133","repository_url":"https://api.github.com/repos/hashicorp/nomad","labels_url":"https://api.github.com/repos/hashicorp/nomad/issues/3133/labels{/name}","comments_url":"https://api.github.com/repos/hashicorp/nomad/issues/3133/comments","events_url":"https://api.github.com/repos/hashicorp/nomad/issues/3133/events","html_url":"https://github.com/hashicorp/nomad/issues/3133","id":253937105,"number":3133,"title":"[unsure, still investigating] significant rise in vault leader CPU usage during and after upgrade to nomad 0.6.2","user":{"login":"wuub","id":588857,"avatar_url":"https://avatars2.githubusercontent.com/u/588857?v=4","gravatar_id":"","url":"https://api.github.com/users/wuub","html_url":"https://github.com/wuub","followers_url":"https://api.github.com/users/wuub/followers","following_url":"https://api.github.com/users/wuub/following{/other_user}","gists_url":"https://api.github.com/users/wuub/gists{/gist_id}","starred_url":"https://api.github.com/users/wuub/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wuub/subscriptions","organizations_url":"https://api.github.com/users/wuub/orgs","repos_url":"https://api.github.com/users/wuub/repos","events_url":"https://api.github.com/users/wuub/events{/privacy}","received_events_url":"https://api.github.com/users/wuub/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":19,"created_at":"2017-08-30T09:37:48Z","updated_at":"2017-09-06T10:09:37Z","closed_at":"2017-09-01T22:04:35Z","author_association":"CONTRIBUTOR","body":"### Nomad version\r\n0.6.2\r\n\r\n### Operating system and Environment details\r\nLinux\r\nVault: 0.7.3\r\n\r\n### Issue\r\nAt some point during our upgrade from nomad 0.5.6 to nomad 0.6.2 (correlation, !causation) we've got an alert about high cpu usage on our **vault** leader. we've since forced different leaders to step-down a several times, and high cpu load follows the leader even after a restart. \r\n\r\n`vault` logs are identical to the ones we were seeing when cpu usage was 10x-20x lower. \r\n\r\nmy weak `strace-fu` and `gdb-fu` did not point to anything obvious.  \r\n\r\n**next step**: we'll try to run one **vault** server with log-level=trace, and force it to assume leader role using a series of step-downs. \r\n\r\nDo you have any ideas about other things we can check that could have such effect on the vault leader?   \r\n\r\n### Reproduction steps\r\nunsure\r\n\r\n### Nomad Server logs (if appropriate)\r\nsilent\r\n\r\n### Nomad Client logs (if appropriate)\r\nnothing vault related\r\n\r\n### Job file (if appropriate)\r\n"},"comment":{"url":"https://api.github.com/repos/hashicorp/nomad/issues/comments/327438928","html_url":"https://github.com/hashicorp/nomad/issues/3133#issuecomment-327438928","issue_url":"https://api.github.com/repos/hashicorp/nomad/issues/3133","id":327438928,"user":{"login":"adamlc","id":809944,"avatar_url":"https://avatars1.githubusercontent.com/u/809944?v=4","gravatar_id":"","url":"https://api.github.com/users/adamlc","html_url":"https://github.com/adamlc","followers_url":"https://api.github.com/users/adamlc/followers","following_url":"https://api.github.com/users/adamlc/following{/other_user}","gists_url":"https://api.github.com/users/adamlc/gists{/gist_id}","starred_url":"https://api.github.com/users/adamlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adamlc/subscriptions","organizations_url":"https://api.github.com/users/adamlc/orgs","repos_url":"https://api.github.com/users/adamlc/repos","events_url":"https://api.github.com/users/adamlc/events{/privacy}","received_events_url":"https://api.github.com/users/adamlc/received_events","type":"User","site_admin":false},"created_at":"2017-09-06T10:09:37Z","updated_at":"2017-09-06T10:09:37Z","author_association":"NONE","body":"@dadgar I've just given 0.6.3-rc1 a spin and I'm still getting the issue with consul-template constantly renewing credentials and issuing lots of SIGHUPs.\r\n\r\n@wuub are you still getting the same issue? Try to determine if I've got a different issue or not!"}}
{"actor":{"display_login":"adamlc"}}