title: RuntimeError: Given groups=1, weight of size [64, 32, 3, 3], expected input[128, 64, 32, 32] to have 32 channels, but got 64 channels instead
body: <p>I am trying to experiment with why we have a Vanishing &amp; exploding gradient, and why <strong>Resnet</strong> is so helpful in avoiding the two problems above. So I decided to train a plain Convolution network with many layers just to know why the model <strong>LOSS</strong> increases as I train with many layers e.g 20 layers. but I am getting this error at some point, I can figure out what might be the issue, but I know it is from my model Architecture.</p>
  <pre><code>images.shape: torch.Size([128, 3, 32, 32])
  ---------------------------------------------------------------------------
  RuntimeError                              Traceback (most recent call last)
  &lt;ipython-input-80-0ad7109b33c1&gt; in &lt;module&gt;
        1 for images, labels in train_dl:
        2     print('images.shape:', images.shape)
  ----&gt; 3     out = model(images)
        4     print('out.shape:', out.shape)
        5     print('out[0]:', out[0])
  
  /opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      725             result = self._slow_forward(*input, **kwargs)
      726         else:
  --&gt; 727             result = self.forward(*input, **kwargs)
      728         for hook in itertools.chain(
      729                 _global_forward_hooks.values(),
  
  &lt;ipython-input-78-81b21c16ed79&gt; in forward(self, xb)
       31 
       32     def forward(self, xb):
  ---&gt; 33         return self.network(xb)
  
  /opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      725             result = self._slow_forward(*input, **kwargs)
      726         else:
  --&gt; 727             result = self.forward(*input, **kwargs)
      728         for hook in itertools.chain(
      729                 _global_forward_hooks.values(),
  
  /opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py in forward(self, input)
      115     def forward(self, input):
      116         for module in self:
  --&gt; 117             input = module(input)
      118         return input
      119 
  
  /opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
      725             result = self._slow_forward(*input, **kwargs)
      726         else:
  --&gt; 727             result = self.forward(*input, **kwargs)
      728         for hook in itertools.chain(
      729                 _global_forward_hooks.values(),
  
  /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py in forward(self, input)
      421 
      422     def forward(self, input: Tensor) -&gt; Tensor:
  --&gt; 423         return self._conv_forward(input, self.weight)
      424 
      425 class Conv3d(_ConvNd):
  
  /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
      418                             _pair(0), self.dilation, self.groups)
      419         return F.conv2d(input, weight, self.bias, self.stride,
  --&gt; 420                         self.padding, self.dilation, self.groups)
      421 
      422     def forward(self, input: Tensor) -&gt; Tensor:
  
  RuntimeError: Given groups=1, weight of size [64, 32, 3, 3], expected input[128, 64, 32, 32] to have 32 channels, but got 64 channels instead
  </code></pre>
  <p>My model <strong>Architecture</strong> is</p>
  <pre><code>class Cifar10CnnModel(ImageClassificationBase):
      def __init__(self):
          super().__init__()
          self.network = nn.Sequential(
              nn.Conv2d(3, 32, kernel_size=3, padding=1),
              nn.Conv2d(32, 32, kernel_size=3, padding=1),
              nn.ReLU(),
              nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),
              nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),
              nn.ReLU(),
              nn.MaxPool2d(2, 2), # output: 64 x 16 x 16
  
              nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1),
              nn.ReLU(),
              nn.Conv2d(128, 128, kernel_size=3, stride=1, padding=1),
              nn.ReLU(),
              nn.MaxPool2d(2, 2), # output: 128 x 8 x 8
  
              nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1),
              nn.ReLU(),
              nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1),
              nn.ReLU(),
              nn.MaxPool2d(2, 2), # output: 256 x 4 x 4
  
              nn.Flatten(), 
              nn.Linear(256*4*4, 1024),
              nn.ReLU(),
              nn.Linear(1024, 512),
              nn.ReLU(),
              nn.Linear(512, 10))
          
      def forward(self, xb):
          return self.network(xb)
  
  for images, labels in train_dl:
      print('images.shape:', images.shape)
      out = model(images)
      print('out.shape:', out.shape)
      print('out[0]:', out[0])
      break
  </code></pre>