HAKE: Human Activity Knowledge Engine
  Human activity understanding is crucial for building automatic intelligent
system. With the help of deep learning, activity understanding has made huge
progress recently. But some challenges such as imbalanced data distribution,
action ambiguity, complex visual patterns still remain. To address these and
promote the activity understanding, we build a large-scale Human Activity
Knowledge Engine (HAKE) based on the human body part states. Upon existing
activity datasets, we annotate the part states of all the active persons in all
images, thus establish the relationship between instance activity and body part
states. Furthermore, we propose a HAKE based part state recognition model with
a knowledge extractor named Activity2Vec and a corresponding part state based
reasoning network. With HAKE, our method can alleviate the learning difficulty
brought by the long-tail data distribution, and bring in interpretability. Now
our HAKE has more than 7 M+ part state annotations and is still under
construction. We first validate our approach on a part of HAKE in this
preliminary paper, where we show 7.2 mAP performance improvement on
Human-Object Interaction recognition, and 12.38 mAP improvement on the one-shot
subsets.
