Deep Learning for Surface Material Classification Using Haptic And
  Visual Information
  When a user scratches a hand-held rigid tool across an object surface, an
acceleration signal can be captured, which carries relevant information about
the surface. More importantly, such a haptic signal is complementary to the
visual appearance of the surface, which suggests the combination of both
modalities for the recognition of the surface material. In this paper, we
present a novel deep learning method dealing with the surface material
classification problem based on a Fully Convolutional Network (FCN), which
takes as input the aforementioned acceleration signal and a corresponding image
of the surface texture. Compared to previous surface material classification
solutions, which rely on a careful design of hand-crafted domain-specific
features, our method automatically extracts discriminative features utilizing
the advanced deep learning methodologies. Experiments performed on the TUM
surface material database demonstrate that our method achieves state-of-the-art
classification accuracy robustly and efficiently.
