HAKE: A Knowledge Engine Foundation for Human Activity Understanding
  Human activity understanding is of widespread interest in artificial
intelligence and spans diverse applications like health care and behavior
analysis. Although there have been advances with deep learning, it remains
challenging. The object recognition-like solutions usually try to map pixels to
semantics directly, but activity patterns are much different from object
patterns, thus hindering another success. In this work, we propose a novel
paradigm to reformulate this task in two-stage: first mapping pixels to an
intermediate space spanned by atomic activity primitives, then programming
detected primitives with interpretable logic rules to infer semantics. To
afford a representative primitive space, we build a knowledge base including
26+ M primitive labels and logic rules from human priors or automatic
discovering. Our framework, Human Activity Knowledge Engine (HAKE), exhibits
superior generalization ability and performance upon canonical methods on
challenging benchmarks. Code and data are available at http://hake-mvig.cn/.
