An Exploration of Active Learning for Affective Digital Phenotyping
  Some of the most severe bottlenecks preventing widespread development of
machine learning models for human behavior include a dearth of labeled training
data and difficulty of acquiring high quality labels. Active learning is a
paradigm for using algorithms to computationally select a useful subset of data
points to label using metrics for model uncertainty and data similarity. We
explore active learning for naturalistic computer vision emotion data, a
particularly heterogeneous and complex data space due to inherently subjective
labels. Using frames collected from gameplay acquired from a therapeutic
smartphone game for children with autism, we run a simulation of active
learning using gameplay prompts as metadata to aid in the active learning
process. We find that active learning using information generated during
gameplay slightly outperforms random selection of the same number of labeled
frames. We next investigate a method to conduct active learning with subjective
data, such as in affective computing, and where multiple crowdsourced labels
can be acquired for each image. Using the Child Affective Facial Expression
(CAFE) dataset, we simulate an active learning process for crowdsourcing many
labels and find that prioritizing frames using the entropy of the crowdsourced
label distribution results in lower categorical cross-entropy loss compared to
random frame selection. Collectively, these results demonstrate pilot
evaluations of two novel active learning approaches for subjective affective
data collected in noisy settings.
