title: How to change activation layer in Pytorch pretrained module?
body: <p>How to change the activation layer of a Pytorch pretrained network? 
  Here is my code : </p>
  
  <pre><code>print("All modules")
  for child in net.children():
      if isinstance(child,nn.ReLU) or isinstance(child,nn.SELU):
          print(child)
  
  print('Before changing activation')
  for child in net.children():
      if isinstance(child,nn.ReLU) or isinstance(child,nn.SELU):
          print(child)
          child=nn.SELU()
          print(child)
  print('after changing activation')
  for child in net.children():
      if isinstance(child,nn.ReLU) or isinstance(child,nn.SELU):
          print(child)
  </code></pre>
  
  <p>Here is my output: </p>
  
  <pre><code>All modules
  ReLU(inplace=True)
  Before changing activation
  ReLU(inplace=True)
  SELU()
  after changing activation
  ReLU(inplace=True)
  </code></pre>