{"action":"created","issue":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/1691","repository_url":"https://api.github.com/repos/pytorch/pytorch","labels_url":"https://api.github.com/repos/pytorch/pytorch/issues/1691/labels{/name}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/1691/comments","events_url":"https://api.github.com/repos/pytorch/pytorch/issues/1691/events","html_url":"https://github.com/pytorch/pytorch/pull/1691","id":232736715,"number":1691,"title":"tensor coalescing dance to avoid copy for cudnn rnn parameters","user":{"login":"jekbradbury","id":11729078,"avatar_url":"https://avatars1.githubusercontent.com/u/11729078?v=3","gravatar_id":"","url":"https://api.github.com/users/jekbradbury","html_url":"https://github.com/jekbradbury","followers_url":"https://api.github.com/users/jekbradbury/followers","following_url":"https://api.github.com/users/jekbradbury/following{/other_user}","gists_url":"https://api.github.com/users/jekbradbury/gists{/gist_id}","starred_url":"https://api.github.com/users/jekbradbury/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jekbradbury/subscriptions","organizations_url":"https://api.github.com/users/jekbradbury/orgs","repos_url":"https://api.github.com/users/jekbradbury/repos","events_url":"https://api.github.com/users/jekbradbury/events{/privacy}","received_events_url":"https://api.github.com/users/jekbradbury/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2017-06-01T00:50:34Z","updated_at":"2017-06-01T20:26:42Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pytorch/pytorch/pulls/1691","html_url":"https://github.com/pytorch/pytorch/pull/1691","diff_url":"https://github.com/pytorch/pytorch/pull/1691.diff","patch_url":"https://github.com/pytorch/pytorch/pull/1691.patch"},"body":"Intended to solve #914, and currently passing tests.\r\nI haven't yet run memory/performance comparisons, but I wanted to see if @apaszke and others are okay with the approach I've taken. I also coalesce on the gradients during the backward pass, which should save a memcpy but doesn't save any memory unless we can get the tensors that will receive the weight gradients to persist between calls."},"comment":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/comments/305610414","html_url":"https://github.com/pytorch/pytorch/pull/1691#issuecomment-305610414","issue_url":"https://api.github.com/repos/pytorch/pytorch/issues/1691","id":305610414,"user":{"login":"jekbradbury","id":11729078,"avatar_url":"https://avatars1.githubusercontent.com/u/11729078?v=3","gravatar_id":"","url":"https://api.github.com/users/jekbradbury","html_url":"https://github.com/jekbradbury","followers_url":"https://api.github.com/users/jekbradbury/followers","following_url":"https://api.github.com/users/jekbradbury/following{/other_user}","gists_url":"https://api.github.com/users/jekbradbury/gists{/gist_id}","starred_url":"https://api.github.com/users/jekbradbury/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jekbradbury/subscriptions","organizations_url":"https://api.github.com/users/jekbradbury/orgs","repos_url":"https://api.github.com/users/jekbradbury/repos","events_url":"https://api.github.com/users/jekbradbury/events{/privacy}","received_events_url":"https://api.github.com/users/jekbradbury/received_events","type":"User","site_admin":false},"created_at":"2017-06-01T20:26:42Z","updated_at":"2017-06-01T20:26:42Z","body":"I forgot that there was repro code in the issue... If you use the repro code with sequence length 20 (about the most that will fit in memory on pytorch master), the memory usage goes from 11433 MiB to 6778 MiB and the time for 100 epochs goes from 63.7 s to 49.7 s. Looks like I really need to figure out how to reuse the gradient buffer, since I'm only getting half the savings we need to say the issue is closed."}}
{"actor":{"display_login":"jekbradbury"}}