title: The result is different when I apply torch.manual_seed before loading cuda() after loading the model
body: <p>I tried to make sure my code to be reproducible (always get the same results)</p>
  
  <p>So I applied below settings before my codes.</p>
  
  <pre><code>os.environ['PYTHONHASHSEED'] = str(args.seed)
  random.seed(args.seed)
  np.random.seed(args.seed)
  torch.manual_seed(args.seed)
  torch.cuda.manual_seed(args.seed)
  torch.cuda.manual_seed_all(args.seed) # if you are using multi-GPU.
  torch.backends.cudnn.benchmark = False
  torch.backends.cudnn.deterministic = True
  </code></pre>
  
  <p>With these settings, I always achieved the same results with the same environment and GPU.</p>
  
  <p>Howerver, when I applied torch.manual_seed() after loading the model.</p>
  
  <pre><code>torch.manual_seed(args.seed)
  model = Net()
  Net.cuda()
  </code></pre>
  
  <pre><code>torch.manual_seed(args.seed)
  model = Net()
  torch.manual_seed(args.seed)
  Net.cuda()
  </code></pre>
  
  <p>The above two results were different.
  How should I understand this situation?</p>
  
  <p>Does seed reinitialize after loading the model?</p>