{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/6215","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/6215/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/6215/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/6215/events","html_url":"https://github.com/fchollet/keras/issues/6215","id":220646402,"number":6215,"title":"Visualizing gradients via TensorBoard","user":{"login":"gokceneraslan","id":1140359,"avatar_url":"https://avatars1.githubusercontent.com/u/1140359?v=3","gravatar_id":"","url":"https://api.github.com/users/gokceneraslan","html_url":"https://github.com/gokceneraslan","followers_url":"https://api.github.com/users/gokceneraslan/followers","following_url":"https://api.github.com/users/gokceneraslan/following{/other_user}","gists_url":"https://api.github.com/users/gokceneraslan/gists{/gist_id}","starred_url":"https://api.github.com/users/gokceneraslan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gokceneraslan/subscriptions","organizations_url":"https://api.github.com/users/gokceneraslan/orgs","repos_url":"https://api.github.com/users/gokceneraslan/repos","events_url":"https://api.github.com/users/gokceneraslan/events{/privacy}","received_events_url":"https://api.github.com/users/gokceneraslan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-04-10T13:35:07Z","updated_at":"2017-04-10T13:54:06Z","closed_at":null,"body":"I'd like to visualize weight grad distributions in TensorBoard. I tried to add grads to TB using `tf.summary.histogram` in TensorBoard class as follows (see full commit [here](https://github.com/gokceneraslan/keras/commit/9649bff94101a2b8eb184ad1f3b6c55b826a6e97)):\r\n\r\n```patch\r\ndiff --git a/keras/callbacks.py b/keras/callbacks.py\r\nindex 49567f9..f605e7d 100644\r\n--- a/keras/callbacks.py\r\n+++ b/keras/callbacks.py\r\n@@ -630,6 +630,9 @@ class TensorBoard(Callback):\r\n \r\n                 for weight in layer.weights:\r\n                     tf.summary.histogram(weight.name, weight)\r\n+                    grads = model.optimizer.get_gradients(model.total_loss,\r\n+                                                          weight)\r\n+                    tf.summary.histogram('{}_grad'.format(weight.name), grads)\r\n                     if self.write_images:\r\n                         w_img = tf.squeeze(weight)\r\n                         shape = w_img.get_shape()\r\n```\r\n\r\nBut it seems targets and sample_weights are defined as placeholders, so they should be specified in the evaluation of grads. Is there a way to do this and add grad visualization to Keras?\r\n\r\nI get the following trace when training starts:\r\n\r\n```\r\nCaused by op 'output_sample_weights', defined at:\r\n  File \"sample.py\", line 9, in <module>\r\n    learning_rate=0.001, reconstruct=False, activation='relu')\r\n  File \"api.py\", line 24, in autoencode\r\n    epochs=epochs, **kwargs)\r\n  File \"train.py\", line 38, in train\r\n    model.compile(loss=loss, optimizer=optimizer)\r\n  File \"/home/goekcen_eraslan/Code/keras-gokcen/keras/engine/training.py\", line 858, in compile\r\n    name=name + '_sample_weights'))\r\n  File \"/home/goekcen_eraslan/Code/keras-gokcen/keras/backend/tensorflow_backend.py\", line 351, in placeholder\r\n    x = tf.placeholder(dtype, shape=shape, name=name)\r\n  File \"/home/goekcen_eraslan/miniconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py\", line 1502, in placeholder\r\n    name=name)\r\n  File \"/home/goekcen_eraslan/miniconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py\", line 2149, in _placeholder\r\n    name=name)\r\n  File \"/home/goekcen_eraslan/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py\", line 763, in apply_op\r\n    op_def=op_def)\r\n  File \"/home/goekcen_eraslan/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py\", line 2327, in create_op\r\n    original_op=self._default_original_op, op_def=op_def)\r\n  File \"/home/goekcen_eraslan/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py\", line 1226, in __init__\r\n    self._traceback = _extract_stack()\r\n\r\nInvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'output_sample_weights' with dtype float\r\n         [[Node: output_sample_weights = Placeholder[dtype=DT_FLOAT, shape=[], _device=\"/job:localhost/replica:0/task:0/gpu:0\"]()]]\r\n```\r\n\r\n- [x] Check that you are up-to-date with the master branch of Keras. You can update with:\r\npip install git+git://github.com/fchollet/keras.git --upgrade --no-deps\r\n\r\n- [x] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found [here](https://www.tensorflow.org/get_started/os_setup).\r\n\r\n- [ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:\r\npip install git+git://github.com/Theano/Theano.git --upgrade --no-deps\r\n\r\n- [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/292955968","html_url":"https://github.com/fchollet/keras/issues/6215#issuecomment-292955968","issue_url":"https://api.github.com/repos/fchollet/keras/issues/6215","id":292955968,"user":{"login":"gokceneraslan","id":1140359,"avatar_url":"https://avatars1.githubusercontent.com/u/1140359?v=3","gravatar_id":"","url":"https://api.github.com/users/gokceneraslan","html_url":"https://github.com/gokceneraslan","followers_url":"https://api.github.com/users/gokceneraslan/followers","following_url":"https://api.github.com/users/gokceneraslan/following{/other_user}","gists_url":"https://api.github.com/users/gokceneraslan/gists{/gist_id}","starred_url":"https://api.github.com/users/gokceneraslan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gokceneraslan/subscriptions","organizations_url":"https://api.github.com/users/gokceneraslan/orgs","repos_url":"https://api.github.com/users/gokceneraslan/repos","events_url":"https://api.github.com/users/gokceneraslan/events{/privacy}","received_events_url":"https://api.github.com/users/gokceneraslan/received_events","type":"User","site_admin":false},"created_at":"2017-04-10T13:54:06Z","updated_at":"2017-04-10T13:54:06Z","body":"Run `examples/mnist_mlp.py` from my Keras repository upon installing patched keras."}}
{"actor":{"display_login":"gokceneraslan"}}