RPT: Toward Transferable Model on Heterogeneous Researcher Data via
  Pre-Training
  With the growth of the academic engines, the mining and analysis acquisition
of massive researcher data, such as collaborator recommendation and researcher
retrieval, has become indispensable. It can improve the quality of services and
intelligence of academic engines. Most of the existing studies for researcher
data mining focus on a single task for a particular application scenario and
learning a task-specific model, which is usually unable to transfer to
out-of-scope tasks. The pre-training technology provides a generalized and
sharing model to capture valuable information from enormous unlabeled data. The
model can accomplish multiple downstream tasks via a few fine-tuning steps. In
this paper, we propose a multi-task self-supervised learning-based researcher
data pre-training model named RPT. Specifically, we divide the researchers'
data into semantic document sets and community graph. We design the
hierarchical Transformer and the local community encoder to capture information
from the two categories of data, respectively. Then, we propose three
self-supervised learning objectives to train the whole model. Finally, we also
propose two transfer modes of RPT for fine-tuning in different scenarios. We
conduct extensive experiments to evaluate RPT, results on three downstream
tasks verify the effectiveness of pre-training for researcher data mining.
