Learning to Identify Object Instances by Touch: Tactile Recognition via
  Multimodal Matching
  Much of the literature on robotic perception focuses on the visual modality.
Vision provides a global observation of a scene, making it broadly useful.
However, in the domain of robotic manipulation, vision alone can sometimes
prove inadequate: in the presence of occlusions or poor lighting, visual object
identification might be difficult. The sense of touch can provide robots with
an alternative mechanism for recognizing objects. In this paper, we study the
problem of touch-based instance recognition. We propose a novel framing of the
problem as multi-modal recognition: the goal of our system is to recognize,
given a visual and tactile observation, whether or not these observations
correspond to the same object. To our knowledge, our work is the first to
address this type of multi-modal instance recognition problem on such a
large-scale with our analysis spanning 98 different objects. We employ a robot
equipped with two GelSight touch sensors, one on each finger, and a
self-supervised, autonomous data collection procedure to collect a dataset of
tactile observations and images. Our experimental results show that it is
possible to accurately recognize object instances by touch alone, including
instances of novel objects that were never seen during training. Our learned
model outperforms other methods on this complex task, including that of human
volunteers.
