Using Sensory Time-cue to enable Unsupervised Multimodal Meta-learning
  As data from IoT (Internet of Things) sensors become ubiquitous,
state-of-the-art machine learning algorithms face many challenges on directly
using sensor data. To overcome these challenges, methods must be designed to
learn directly from sensors without manual annotations. This paper introduces
Sensory Time-cue for Unsupervised Meta-learning (STUM). Different from
traditional learning approaches that either heavily depend on labels or on
time-independent feature extraction assumptions, such as Gaussian distribution
features, the STUM system uses time relation of inputs to guide the feature
space formation within and across modalities. The fact that STUM learns from a
variety of small tasks may put this method in the camp of Meta-Learning.
Different from existing Meta-Learning approaches, STUM learning tasks are
composed within and across multiple modalities based on time-cue co-exist with
the IoT streaming data. In an audiovisual learning example, because consecutive
visual frames usually comprise the same object, this approach provides a unique
way to organize features from the same object together. The same method can
also organize visual object features with the object's spoken-name features
together if the spoken name is presented with the object at about the same
time. This cross-modality feature organization may further help the
organization of visual features that belong to similar objects but acquired at
different location and time. Promising results are achieved through
evaluations.
