{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/5592","repository_url":"https://api.github.com/repos/dmlc/mxnet","labels_url":"https://api.github.com/repos/dmlc/mxnet/issues/5592/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/5592/comments","events_url":"https://api.github.com/repos/dmlc/mxnet/issues/5592/events","html_url":"https://github.com/dmlc/mxnet/issues/5592","id":217425679,"number":5592,"title":"[Discussion] CreateBackwardOp interface for `Operator`","user":{"login":"jermainewang","id":4057701,"avatar_url":"https://avatars2.githubusercontent.com/u/4057701?v=3","gravatar_id":"","url":"https://api.github.com/users/jermainewang","html_url":"https://github.com/jermainewang","followers_url":"https://api.github.com/users/jermainewang/followers","following_url":"https://api.github.com/users/jermainewang/following{/other_user}","gists_url":"https://api.github.com/users/jermainewang/gists{/gist_id}","starred_url":"https://api.github.com/users/jermainewang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jermainewang/subscriptions","organizations_url":"https://api.github.com/users/jermainewang/orgs","repos_url":"https://api.github.com/users/jermainewang/repos","events_url":"https://api.github.com/users/jermainewang/events{/privacy}","received_events_url":"https://api.github.com/users/jermainewang/received_events","type":"User","site_admin":false},"labels":[{"id":272547979,"url":"https://api.github.com/repos/dmlc/mxnet/labels/discussion","name":"discussion","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2017-03-28T01:58:18Z","updated_at":"2017-03-28T14:37:52Z","closed_at":null,"body":"Based on discussion in https://github.com/dmlc/nnvm/pull/108 , we propose two new interfaces for `Operator` class:\r\n```c++\r\n// Return whether the operator is stateless, which means the backward operator can be created\r\n// without forward operator.\r\nvirtual bool IsStateless() const {\r\n  return false;\r\n}\r\n// [Optional] Create an operator that is used to do backward.\r\n// It defaults to return null, which means the backward operator cannot be created\r\n// standalone and should be created together with forward operator.\r\nvirtual Operator* CreateBackwardOperatorEx(\r\n    const Context& ctx,\r\n    const std::vector<TShape>& in_shape,\r\n    const std::vector<int>& in_type,\r\n    const std::vector<TShape>& out_shape,\r\n    const std::vector<int>& out_type) const {\r\n  return nullptr;\r\n}\r\n```\r\n\r\nBy default, these interfaces return stateful for all current operators. Upon creating `OpExecutor` for a node, following logic will be used:\r\n* If it is a forward node, go straight create its `OpExecutor`. (same as original logic)\r\n* If it is a backward node and its forward node exists, reuse its forward `OpExecutor`. (same as original logic)\r\n* If it is a backward node but no forward node exists, create the `OpExecutor` using the new interface. (the new logic)\r\n\r\nTherefore, **nothing will be changed for all current MXNet codes**. The only part that uses this currently is `AutogradRuntime`. It will first check whether the given symbol is stateless (if not, raise error). Then it will use the third logic above to create its backward `OpExecutor`.\r\n\r\nThe implementation overhead is to override above interfaces for stateless operators. This should be easy since most of the current operator is stateless so the `CreateBackwardOperatorEx` usually follows the *same* logic as `CreateOperatorEx`.\r\n\r\nPlease leave your comments.\r\n\r\n@piiswrong @tqchen @ZihengJiang "},"comment":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/comments/289791037","html_url":"https://github.com/dmlc/mxnet/issues/5592#issuecomment-289791037","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/5592","id":289791037,"user":{"login":"jermainewang","id":4057701,"avatar_url":"https://avatars2.githubusercontent.com/u/4057701?v=3","gravatar_id":"","url":"https://api.github.com/users/jermainewang","html_url":"https://github.com/jermainewang","followers_url":"https://api.github.com/users/jermainewang/followers","following_url":"https://api.github.com/users/jermainewang/following{/other_user}","gists_url":"https://api.github.com/users/jermainewang/gists{/gist_id}","starred_url":"https://api.github.com/users/jermainewang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jermainewang/subscriptions","organizations_url":"https://api.github.com/users/jermainewang/orgs","repos_url":"https://api.github.com/users/jermainewang/repos","events_url":"https://api.github.com/users/jermainewang/events{/privacy}","received_events_url":"https://api.github.com/users/jermainewang/received_events","type":"User","site_admin":false},"created_at":"2017-03-28T14:37:52Z","updated_at":"2017-03-28T14:37:52Z","body":"Oops, mistake. It is not in the `Operator` class. The interface should be in the `OperatorProperty` class."}}
{"actor":{"display_login":"jermainewang"}}