Co-training for Deep Object Detection: Comparing Single-modal and
  Multi-modal Approaches
  Top-performing computer vision models are powered by convolutional neural
networks (CNNs). Training an accurate CNN highly depends on both the raw sensor
data and their associated ground truth (GT). Collecting such GT is usually done
through human labeling, which is time-consuming and does not scale as we wish.
This data labeling bottleneck may be intensified due to domain shifts among
image sensors, which could force per-sensor data labeling. In this paper, we
focus on the use of co-training, a semi-supervised learning (SSL) method, for
obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep
object detectors. In particular, we assess the goodness of multi-modal
co-training by relying on two different views of an image, namely, appearance
(RGB) and estimated depth (D). Moreover, we compare appearance-based
single-modal co-training with multi-modal. Our results suggest that in a
standard SSL setting (no domain shift, a few human-labeled data) and under
virtual-to-real domain shift (many virtual-world labeled data, no human-labeled
data) multi-modal co-training outperforms single-modal. In the latter case, by
performing GAN-based domain translation both co-training modalities are on
pair; at least, when using an off-the-shelf depth estimation model not
specifically trained on the translated images.
