{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/10449","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/10449/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/10449/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/10449/events","html_url":"https://github.com/tensorflow/tensorflow/issues/10449","id":233783549,"number":10449,"title":"[Feature]Adding automatic model average parallelism support in TF","user":{"login":"yangjunpro","id":407784,"avatar_url":"https://avatars3.githubusercontent.com/u/407784?v=4","gravatar_id":"","url":"https://api.github.com/users/yangjunpro","html_url":"https://github.com/yangjunpro","followers_url":"https://api.github.com/users/yangjunpro/followers","following_url":"https://api.github.com/users/yangjunpro/following{/other_user}","gists_url":"https://api.github.com/users/yangjunpro/gists{/gist_id}","starred_url":"https://api.github.com/users/yangjunpro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yangjunpro/subscriptions","organizations_url":"https://api.github.com/users/yangjunpro/orgs","repos_url":"https://api.github.com/users/yangjunpro/repos","events_url":"https://api.github.com/users/yangjunpro/events{/privacy}","received_events_url":"https://api.github.com/users/yangjunpro/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"sherrym","id":12770037,"avatar_url":"https://avatars0.githubusercontent.com/u/12770037?v=4","gravatar_id":"","url":"https://api.github.com/users/sherrym","html_url":"https://github.com/sherrym","followers_url":"https://api.github.com/users/sherrym/followers","following_url":"https://api.github.com/users/sherrym/following{/other_user}","gists_url":"https://api.github.com/users/sherrym/gists{/gist_id}","starred_url":"https://api.github.com/users/sherrym/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sherrym/subscriptions","organizations_url":"https://api.github.com/users/sherrym/orgs","repos_url":"https://api.github.com/users/sherrym/repos","events_url":"https://api.github.com/users/sherrym/events{/privacy}","received_events_url":"https://api.github.com/users/sherrym/received_events","type":"User","site_admin":false},"assignees":[{"login":"sherrym","id":12770037,"avatar_url":"https://avatars0.githubusercontent.com/u/12770037?v=4","gravatar_id":"","url":"https://api.github.com/users/sherrym","html_url":"https://github.com/sherrym","followers_url":"https://api.github.com/users/sherrym/followers","following_url":"https://api.github.com/users/sherrym/following{/other_user}","gists_url":"https://api.github.com/users/sherrym/gists{/gist_id}","starred_url":"https://api.github.com/users/sherrym/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sherrym/subscriptions","organizations_url":"https://api.github.com/users/sherrym/orgs","repos_url":"https://api.github.com/users/sherrym/repos","events_url":"https://api.github.com/users/sherrym/events{/privacy}","received_events_url":"https://api.github.com/users/sherrym/received_events","type":"User","site_admin":false}],"milestone":null,"comments":6,"created_at":"2017-06-06T05:33:29Z","updated_at":"2017-07-19T02:09:31Z","closed_at":null,"body":"Model Average is a common paradigm for distributed DL training, also there are several papers regarding to it and its variants:\r\n\r\n(https://arxiv.org/pdf/1410.7455v8.pdf)\r\n(http://www.microsoft.com/en-us/research/wp-content/uploads/2016/08/0005880.pdf)\r\n\r\nDo we have any plan to bring this support into TF? Actually we have already implemented model average support and benchmarked it on several in-house models, the speed up is good, about 3X(4 cards) to 9X(16 cards) convergence speed-up. Also we are working on making the model average mechanism as automatic as possible, so modeling guys could easily leverage this nice feature.\r\n\r\nIf TF community is fine with this feature/enhancement, we will be happy to merge our code into community.\r\n\r\nThanks.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/316251719","html_url":"https://github.com/tensorflow/tensorflow/issues/10449#issuecomment-316251719","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/10449","id":316251719,"user":{"login":"hellolovetiger","id":18715195,"avatar_url":"https://avatars0.githubusercontent.com/u/18715195?v=4","gravatar_id":"","url":"https://api.github.com/users/hellolovetiger","html_url":"https://github.com/hellolovetiger","followers_url":"https://api.github.com/users/hellolovetiger/followers","following_url":"https://api.github.com/users/hellolovetiger/following{/other_user}","gists_url":"https://api.github.com/users/hellolovetiger/gists{/gist_id}","starred_url":"https://api.github.com/users/hellolovetiger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hellolovetiger/subscriptions","organizations_url":"https://api.github.com/users/hellolovetiger/orgs","repos_url":"https://api.github.com/users/hellolovetiger/repos","events_url":"https://api.github.com/users/hellolovetiger/events{/privacy}","received_events_url":"https://api.github.com/users/hellolovetiger/received_events","type":"User","site_admin":false},"created_at":"2017-07-19T02:09:31Z","updated_at":"2017-07-19T02:09:31Z","body":"Cool, thanks, @yangjunpro ."}}
{"actor":{"display_login":"hellolovetiger"}}