KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization
  and Completion
  A comprehensive knowledge graph (KG) contains an instance-level entity graph
and an ontology-level concept graph. The two-view KG provides a testbed for
models to "simulate" human's abilities on knowledge abstraction,
concretization, and completion (KACC), which are crucial for human to recognize
the world and manage learned knowledge. Existing studies mainly focus on
partial aspects of KACC. In order to promote thorough analyses for KACC
abilities of models, we propose a unified KG benchmark by improving existing
benchmarks in terms of dataset scale, task coverage, and difficulty.
Specifically, we collect new datasets that contain larger concept graphs,
abundant cross-view links as well as dense entity graphs. Based on the
datasets, we propose novel tasks such as multi-hop knowledge abstraction (MKA),
multi-hop knowledge concretization (MKC) and then design a comprehensive
benchmark. For MKA and MKC tasks, we further annotate multi-hop hierarchical
triples as harder samples. The experimental results of existing methods
demonstrate the challenges of our benchmark. The resource is available at
https://github.com/thunlp/KACC.
