{"action":"created","issue":{"url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava/issues/64","repository_url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava","labels_url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava/issues/64/comments","events_url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava/issues/64/events","html_url":"https://github.com/AstraZeneca-NGS/VarDictJava/issues/64","id":206312179,"number":64,"title":"Improving memory usage over small regions","user":{"login":"chapmanb","id":39391,"avatar_url":"https://avatars.githubusercontent.com/u/39391?v=3","gravatar_id":"","url":"https://api.github.com/users/chapmanb","html_url":"https://github.com/chapmanb","followers_url":"https://api.github.com/users/chapmanb/followers","following_url":"https://api.github.com/users/chapmanb/following{/other_user}","gists_url":"https://api.github.com/users/chapmanb/gists{/gist_id}","starred_url":"https://api.github.com/users/chapmanb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chapmanb/subscriptions","organizations_url":"https://api.github.com/users/chapmanb/orgs","repos_url":"https://api.github.com/users/chapmanb/repos","events_url":"https://api.github.com/users/chapmanb/events{/privacy}","received_events_url":"https://api.github.com/users/chapmanb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-02-08T20:17:58Z","updated_at":"2017-02-09T08:41:11Z","closed_at":null,"body":"Zaal, Zhongwu and all;\r\nWe're seeing some cases where VarDict has higher than expected memory usage on relatively small regions. Here is a self contained example over a 1.5Mb region where VarDict memory usage spikes at ~6Gb:\r\n```\r\nwget https://s3.amazonaws.com/chapmanb/testcases/vardict_debug_memory.tar.gz\r\n```\r\nWe've been seeing issues where these type of memory spikes cause schedulers to kill jobs, requiring allocating large amounts of memory (~10Gb+/core) to VarDict jobs. Allocating this much memory runs less VarDict jobs per machine to work around the issue, but is sub-optimal since it reduces the total number of jobs run concurrently.\r\n\r\nDo you have any insight into why memory usage gets so high over this region? Are there any tweaks we can do to VarDict, or region selection, to avoid this? Thanks for any thoughts or directions to pursue."},"comment":{"url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava/issues/comments/278579854","html_url":"https://github.com/AstraZeneca-NGS/VarDictJava/issues/64#issuecomment-278579854","issue_url":"https://api.github.com/repos/AstraZeneca-NGS/VarDictJava/issues/64","id":278579854,"user":{"login":"mjafin","id":3286525,"avatar_url":"https://avatars.githubusercontent.com/u/3286525?v=3","gravatar_id":"","url":"https://api.github.com/users/mjafin","html_url":"https://github.com/mjafin","followers_url":"https://api.github.com/users/mjafin/followers","following_url":"https://api.github.com/users/mjafin/following{/other_user}","gists_url":"https://api.github.com/users/mjafin/gists{/gist_id}","starred_url":"https://api.github.com/users/mjafin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mjafin/subscriptions","organizations_url":"https://api.github.com/users/mjafin/orgs","repos_url":"https://api.github.com/users/mjafin/repos","events_url":"https://api.github.com/users/mjafin/events{/privacy}","received_events_url":"https://api.github.com/users/mjafin/received_events","type":"User","site_admin":false},"created_at":"2017-02-09T08:41:11Z","updated_at":"2017-02-09T08:41:11Z","body":"Might be worth doing a quick check with realignment switched off if that is the reason"}}
{"actor":{"display_login":"mjafin"}}