{"action":"created","issue":{"url":"https://api.github.com/repos/tsirif/platoon/issues/3","repository_url":"https://api.github.com/repos/tsirif/platoon","labels_url":"https://api.github.com/repos/tsirif/platoon/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/tsirif/platoon/issues/3/comments","events_url":"https://api.github.com/repos/tsirif/platoon/issues/3/events","html_url":"https://github.com/tsirif/platoon/pull/3","id":212911228,"number":3,"title":"Fix/multi node new PR","user":{"login":"cshanbo","id":9649941,"avatar_url":"https://avatars0.githubusercontent.com/u/9649941?v=3","gravatar_id":"","url":"https://api.github.com/users/cshanbo","html_url":"https://github.com/cshanbo","followers_url":"https://api.github.com/users/cshanbo/followers","following_url":"https://api.github.com/users/cshanbo/following{/other_user}","gists_url":"https://api.github.com/users/cshanbo/gists{/gist_id}","starred_url":"https://api.github.com/users/cshanbo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cshanbo/subscriptions","organizations_url":"https://api.github.com/users/cshanbo/orgs","repos_url":"https://api.github.com/users/cshanbo/repos","events_url":"https://api.github.com/users/cshanbo/events{/privacy}","received_events_url":"https://api.github.com/users/cshanbo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-03-09T01:43:18Z","updated_at":"2017-03-29T21:38:16Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/tsirif/platoon/pulls/3","html_url":"https://github.com/tsirif/platoon/pull/3","diff_url":"https://github.com/tsirif/platoon/pull/3.diff","patch_url":"https://github.com/tsirif/platoon/pull/3.patch"},"body":"1. Fix importing error\r\n2. Fix unittest\r\n    1. as I said in the email I sent to you, the `worker.close()` in `tearDownClass` and `__exit__()` in `worker` close the socket twice, and cause an error. We only need `close` once.\r\n3. Fix argument in lstm example\r\n"},"comment":{"url":"https://api.github.com/repos/tsirif/platoon/issues/comments/290234064","html_url":"https://github.com/tsirif/platoon/pull/3#issuecomment-290234064","issue_url":"https://api.github.com/repos/tsirif/platoon/issues/3","id":290234064,"user":{"login":"tsirif","id":4084993,"avatar_url":"https://avatars3.githubusercontent.com/u/4084993?v=3","gravatar_id":"","url":"https://api.github.com/users/tsirif","html_url":"https://github.com/tsirif","followers_url":"https://api.github.com/users/tsirif/followers","following_url":"https://api.github.com/users/tsirif/following{/other_user}","gists_url":"https://api.github.com/users/tsirif/gists{/gist_id}","starred_url":"https://api.github.com/users/tsirif/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tsirif/subscriptions","organizations_url":"https://api.github.com/users/tsirif/orgs","repos_url":"https://api.github.com/users/tsirif/repos","events_url":"https://api.github.com/users/tsirif/events{/privacy}","received_events_url":"https://api.github.com/users/tsirif/received_events","type":"User","site_admin":false},"created_at":"2017-03-29T21:38:16Z","updated_at":"2017-03-29T21:38:16Z","body":"Hello there Shanbo,\r\n\r\nI suspect that the `import` order was out of place. Resulting in a faulty initialization of Theano module when initializing Platoon. There is no need to change the MPI method to `Spawn_multiple` for dynamic process managing. I think we can use something else to work around this and make import order a trivial matter.\r\n\r\nWhen a worker is [initialized](https://github.com/cshanbo/platoon/blob/fix/multi-node/platoon/channel/worker.py#L298), it asks its node controller to return which device it shall use. Then currently, it sets its own environment and imports Theano. The goal is to have Theano, create a GPU **context** object for the correct GPU and use this to initialize NCCL through pygpu. I think we can skip that bad practice of setting its own environment and use [this](https://github.com/cshanbo/platoon/blob/fix/multi-node/platoon/channel/worker.py#L298) Theano method to setup the device whenever a Platoon worker decides to register.\r\n\r\ni.e.\r\n1. Ask controller to return a worker's GPU device name (already done in worker's 298)\r\n2. Import Theano\r\n3. `theanoga.use(self.device)`\r\n4. and then let lines 306 onwards as they have.\r\n\r\nThis should resolve the issue and make worker's code independent of import order."}}
{"actor":{"display_login":"tsirif"}}