{"action":"created","issue":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/1392","repository_url":"https://api.github.com/repos/BD2KGenomics/toil","labels_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/1392/labels{/name}","comments_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/1392/comments","events_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/1392/events","html_url":"https://github.com/BD2KGenomics/toil/pull/1392","id":195719525,"number":1392,"title":"Plugin batch systems","user":{"login":"evan-wehi","id":22020877,"avatar_url":"https://avatars0.githubusercontent.com/u/22020877?v=3","gravatar_id":"","url":"https://api.github.com/users/evan-wehi","html_url":"https://github.com/evan-wehi","followers_url":"https://api.github.com/users/evan-wehi/followers","following_url":"https://api.github.com/users/evan-wehi/following{/other_user}","gists_url":"https://api.github.com/users/evan-wehi/gists{/gist_id}","starred_url":"https://api.github.com/users/evan-wehi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/evan-wehi/subscriptions","organizations_url":"https://api.github.com/users/evan-wehi/orgs","repos_url":"https://api.github.com/users/evan-wehi/repos","events_url":"https://api.github.com/users/evan-wehi/events{/privacy}","received_events_url":"https://api.github.com/users/evan-wehi/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":37,"created_at":"2016-12-15T05:21:53Z","updated_at":"2017-06-30T20:02:32Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/BD2KGenomics/toil/pulls/1392","html_url":"https://github.com/BD2KGenomics/toil/pull/1392","diff_url":"https://github.com/BD2KGenomics/toil/pull/1392.diff","patch_url":"https://github.com/BD2KGenomics/toil/pull/1392.patch"},"body":"This supersedes \"Allow dynamic addition of new BatchSystem (#1374)\"\r\n\r\nThis changes allows the registration of new implementations of AbstractBatchSystem. There is a registry for factories to create new BatchSystems (toil.batchSystems.registry) and a seperate registry for adding command line options (toil.batchSystems.options). The registries are pre-populated with the exiting BatchSystem implementations. The reason for separating options and factories is that command line options can be present even if the dependencies for the batch system are not. (Arguably, you might not want command line options present if the can't be used).\r\n\r\nThe motivating use case is for sites (or even individual users) to be able to add BatchSystems that can implement local policies or to implement BatchSystems that are not generally useful. It also decouples the concrete BatchSystem implementations from common.py.\r\n\r\nNew BatchSystem can be added programmatically as follows:\r\n\r\n```\r\nfrom toil.batchSystems.registry import addBatchSystemFactory\r\nfrom toil.batchSystems.options import addOptionsDefinition\r\n    \r\ndef drmaaBatchSystemFactory():\r\n    from wehi_pipeline.batchSystems.drmaaBatchSystem import DrmaaBatchSystem\r\n    return DrmaaBatchSystem\r\n    \r\naddBatchSystemFactory('drmaa', drmaaBatchSystemFactory)\r\n       \r\ndef addOptions(addOptionFn):\r\n    addOptionFn(\"--jobQueue\", dest=\"jobQueue\", default=None,\r\n            help=(\"A job queue (used by the DRMAA batch system\"))\r\n        \r\naddOptionsDefinition(addOptions)\r\n```"},"comment":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/comments/312360720","html_url":"https://github.com/BD2KGenomics/toil/pull/1392#issuecomment-312360720","issue_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/1392","id":312360720,"user":{"login":"ejacox","id":21335787,"avatar_url":"https://avatars1.githubusercontent.com/u/21335787?v=3","gravatar_id":"","url":"https://api.github.com/users/ejacox","html_url":"https://github.com/ejacox","followers_url":"https://api.github.com/users/ejacox/followers","following_url":"https://api.github.com/users/ejacox/following{/other_user}","gists_url":"https://api.github.com/users/ejacox/gists{/gist_id}","starred_url":"https://api.github.com/users/ejacox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ejacox/subscriptions","organizations_url":"https://api.github.com/users/ejacox/orgs","repos_url":"https://api.github.com/users/ejacox/repos","events_url":"https://api.github.com/users/ejacox/events{/privacy}","received_events_url":"https://api.github.com/users/ejacox/received_events","type":"User","site_admin":false},"created_at":"2017-06-30T20:02:32Z","updated_at":"2017-06-30T20:02:32Z","body":"Thank you for the effort @evan-wehi. I will rerun the test. If I have time, I will try to take a look at the code again today. Otherwise, I should have some time on Monday."}}
{"actor":{"display_login":"ejacox"}}