Convolutional Gated MLP: Combining Convolutions & gMLP
  To the best of our knowledge, this is the first paper to introduce
Convolutions to Gated MultiLayer Perceptron and contributes an implementation
of this novel Deep Learning architecture. Google Brain introduced the gMLP in
May 2021. Microsoft introduced Convolutions in Vision Transformer in Mar 2021.
Inspired by both gMLP and CvT, we introduce convolutional layers in gMLP. CvT
combined the power of Convolutions and Attention. Our implementation combines
the best of Convolutional learning along with spatial gated MLP. Further, the
paper visualizes how CgMLP learns. Visualizations show how CgMLP learns from
features such as outline of a car. While Attention was the basis of much of
recent progress in Deep Learning, gMLP proposed an approach that doesn't use
Attention computation. In Transformer based approaches, a whole lot of
Attention matrixes need to be learnt using vast amount of training data. In
gMLP, the fine tunning for new tasks can be challenging by transfer learning
with smaller datasets. We implement CgMLP and compares it with gMLP on CIFAR
dataset. Experimental results explore the power of generaliza-tion of CgMLP,
while gMLP tend to drastically overfit the training data.
  To summarize, the paper contributes a novel Deep Learning architecture and
demonstrates the learning mechanism of CgMLP through visualizations, for the
first time in literature.
