title: PyTorch autograd -- grad can be implicitly created only for scalar outputs
body: <p>I am using the <code>autograd</code> tool in <code>PyTorch</code>, and have found myself in a situation where I need to access the values in a 1D tensor by means of an integer index. Something like this:</p>
  
  <pre><code>def basic_fun(x_cloned):
      res = []
      for i in range(len(x)):
          res.append(x_cloned[i] * x_cloned[i])
      print(res)
      return Variable(torch.FloatTensor(res))
  
  
  def get_grad(inp, grad_var):
      A = basic_fun(inp)
      A.backward()
      return grad_var.grad
  
  
  x = Variable(torch.FloatTensor([1, 2, 3, 4, 5]), requires_grad=True)
  x_cloned = x.clone()
  print(get_grad(x_cloned, x))
  </code></pre>
  
  <p>I am getting the following error message:</p>
  
  <pre><code>[tensor(1., grad_fn=&lt;ThMulBackward&gt;), tensor(4., grad_fn=&lt;ThMulBackward&gt;), tensor(9., grad_fn=&lt;ThMulBackward&gt;), tensor(16., grad_fn=&lt;ThMulBackward&gt;), tensor(25., grad_fn=&lt;ThMulBackward&gt;)]
  Traceback (most recent call last):
    File "/home/mhy/projects/pytorch-optim/predict.py", line 74, in &lt;module&gt;
      print(get_grad(x_cloned, x))
    File "/home/mhy/projects/pytorch-optim/predict.py", line 68, in get_grad
      A.backward()
    File "/home/mhy/.local/lib/python3.5/site-packages/torch/tensor.py", line 93, in backward
      torch.autograd.backward(self, gradient, retain_graph, create_graph)
    File "/home/mhy/.local/lib/python3.5/site-packages/torch/autograd/__init__.py", line 90, in backward
      allow_unreachable=True)  # allow_unreachable flag
  RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
  </code></pre>
  
  <p>I am in general, a bit skeptical about how using the cloned version of a variable is supposed to keep that variable in gradient computation. The variable itself is effectively not used in the computation of <code>A</code>, and so when you call <code>A.backward()</code>, it should not be part of that operation. </p>
  
  <p>I appreciate your help with this approach or if there is a better way to avoid losing the gradient history and still index through a 1D tensor with <code>requires_grad=True</code>!</p>
  
  <h2>**Edit (September 15):**</h2>
  
  <p><code>res</code> is a list of zero-dimensional tensors containing squared values of 1 to 5. To concatenate in a single tensor containing [1.0, 4.0, ..., 25.0], I changed <code>return Variable(torch.FloatTensor(res))</code> to <code>torch.stack(res, dim=0)</code>, which produces <code>tensor([ 1.,  4.,  9., 16., 25.], grad_fn=&lt;StackBackward&gt;)</code>.</p>
  
  <p>However, I am getting this new error, caused by the <code>A.backward()</code> line.</p>
  
  <pre><code>Traceback (most recent call last):
    File "&lt;project_path&gt;/playground.py", line 22, in &lt;module&gt;
      print(get_grad(x_cloned, x))
    File "&lt;project_path&gt;/playground.py", line 16, in get_grad
      A.backward()
    File "/home/mhy/.local/lib/python3.5/site-packages/torch/tensor.py", line 93, in backward
      torch.autograd.backward(self, gradient, retain_graph, create_graph)
    File "/home/mhy/.local/lib/python3.5/site-packages/torch/autograd/__init__.py", line 84, in backward
      grad_tensors = _make_grads(tensors, grad_tensors)
    File "/home/mhy/.local/lib/python3.5/site-packages/torch/autograd/__init__.py", line 28, in _make_grads
      raise RuntimeError("grad can be implicitly created only for scalar outputs")
  RuntimeError: grad can be implicitly created only for scalar outputs
  </code></pre>