{"action":"created","issue":{"url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow/issues/37","repository_url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow","labels_url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow/issues/37/labels{/name}","comments_url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow/issues/37/comments","events_url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow/issues/37/events","html_url":"https://github.com/naturomics/CapsNet-Tensorflow/issues/37","id":276195077,"number":37,"title":"b_IJ update","user":{"login":"erlebach","id":324708,"avatar_url":"https://avatars3.githubusercontent.com/u/324708?v=4","gravatar_id":"","url":"https://api.github.com/users/erlebach","html_url":"https://github.com/erlebach","followers_url":"https://api.github.com/users/erlebach/followers","following_url":"https://api.github.com/users/erlebach/following{/other_user}","gists_url":"https://api.github.com/users/erlebach/gists{/gist_id}","starred_url":"https://api.github.com/users/erlebach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erlebach/subscriptions","organizations_url":"https://api.github.com/users/erlebach/orgs","repos_url":"https://api.github.com/users/erlebach/repos","events_url":"https://api.github.com/users/erlebach/events{/privacy}","received_events_url":"https://api.github.com/users/erlebach/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-11-22T20:26:04Z","updated_at":"2017-11-23T13:56:35Z","closed_at":null,"author_association":"NONE","body":"Hi, \r\nin CapsLayer.py, consider the current code. One sees that if cfg.iter_routing == 1, that b_IJ never gets updated. Surely that is not the intent? Shouldn't b_IJ be updated at every iteration of the routing? Thanks.\r\n   \r\nGordon\r\n\r\n\r\n\r\n```\r\nif r_iter == cfg.iter_routing - 1:\r\n                # line 5:\r\n                # weighting u_hat with c_IJ, element-wise in the last two dims\r\n                # => [batch_size, 1152, 10, 16, 1]\r\n                s_J = tf.multiply(c_IJ, u_hat)\r\n                # then sum in the second dim, resulting in [batch_size, 1, 10, 16, 1]\r\n                s_J = tf.reduce_sum(s_J, axis=1, keep_dims=True)\r\n                assert s_J.get_shape() == [cfg.batch_size, 1, 10, 16, 1]\r\n\r\n                # line 6:\r\n                # squash using Eq.1,\r\n                v_J = squash(s_J)\r\n                assert v_J.get_shape() == [cfg.batch_size, 1, 10, 16, 1]\r\n            elif r_iter < cfg.iter_routing - 1:  # Inner iterations, do not apply backpropagation\r\n                s_J = tf.multiply(c_IJ, u_hat_stopped)\r\n                s_J = tf.reduce_sum(s_J, axis=1, keep_dims=True)\r\n                v_J = squash(s_J)         # <<<<<<<< MISSING UPDATE of B_IJ? \r\n\r\n                # line 7:\r\n                # reshape & tile v_j from [batch_size ,1, 10, 16, 1] to [batch_size, 1152, 10, 16, 1]\r\n                # then matmul in the last tow dim: [16, 1].T x [16, 1] => [1, 1], reduce mean in the\r\n                # batch_size dim, resulting in [1, 1152, 10, 1, 1]\r\n                v_J_tiled = tf.tile(v_J, [1, 1152, 1, 1, 1])\r\n                u_produce_v = tf.matmul(u_hat_stopped, v_J_tiled, transpose_a=True)\r\n                assert u_produce_v.get_shape() == [cfg.batch_size, 1152, 10, 1, 1]\r\n\r\n                # b_IJ += tf.reduce_sum(u_produce_v, axis=0, keep_dims=True)\r\n                b_IJ += u_produce_v   # <<<<<< PERHAPS THIS LINE SHOULD BE OUTSIDE THE r_iter LOOP? \r\n```"},"comment":{"url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow/issues/comments/346624886","html_url":"https://github.com/naturomics/CapsNet-Tensorflow/issues/37#issuecomment-346624886","issue_url":"https://api.github.com/repos/naturomics/CapsNet-Tensorflow/issues/37","id":346624886,"user":{"login":"XifengGuo","id":20176064,"avatar_url":"https://avatars2.githubusercontent.com/u/20176064?v=4","gravatar_id":"","url":"https://api.github.com/users/XifengGuo","html_url":"https://github.com/XifengGuo","followers_url":"https://api.github.com/users/XifengGuo/followers","following_url":"https://api.github.com/users/XifengGuo/following{/other_user}","gists_url":"https://api.github.com/users/XifengGuo/gists{/gist_id}","starred_url":"https://api.github.com/users/XifengGuo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/XifengGuo/subscriptions","organizations_url":"https://api.github.com/users/XifengGuo/orgs","repos_url":"https://api.github.com/users/XifengGuo/repos","events_url":"https://api.github.com/users/XifengGuo/events{/privacy}","received_events_url":"https://api.github.com/users/XifengGuo/received_events","type":"User","site_admin":false},"created_at":"2017-11-23T13:56:35Z","updated_at":"2017-11-23T13:56:35Z","author_association":"COLLABORATOR","body":"@erlebach I think this is consistent with the paper. `iter_routing=1` should represent not using routing at all. In Figure A.1, they should have done an experiment on `iter_routing=1`, if otherwise. And they should have reported result with `iter_routing=1` in Table 1 to validate the effectiveness of routing algorithm."}}
{"actor":{"display_login":"XifengGuo"}}