Reproducing BowNet: Learning Representations by Predicting Bags of
  Visual Words
  This work aims to reproduce results from the CVPR 2020 paper by Gidaris et
al. Self-supervised learning (SSL) is used to learn feature representations of
an image using an unlabeled dataset. This work proposes to use bag-of-words
(BoW) deep feature descriptors as a self-supervised learning target to learn
robust, deep representations. BowNet is trained to reconstruct the histogram of
visual words (ie. the deep BoW descriptor) of a reference image when presented
a perturbed version of the image as input. Thus, this method aims to learn
perturbation-invariant and context-aware image features that can be useful for
few-shot tasks or supervised downstream tasks. In the paper, the author
describes BowNet as a network consisting of a convolutional feature extractor
$\Phi(\cdot)$ and a Dense-softmax layer $\Omega(\cdot)$ trained to predict BoW
features from images. After BoW training, the features of $\Phi$ are used in
downstream tasks. For this challenge we were trying to build and train a
network that could reproduce the CIFAR-100 accuracy improvements reported in
the original paper. However, we were unsuccessful in reproducing an accuracy
improvement comparable to what the authors mentioned. This could be for a
variety of factors and we believe that time constraints were the primary
bottleneck.
