Comprehensive Analysis of Negative Sampling in Knowledge Graph Representation Learning
Negative sampling (NS) loss plays an important role in learning knowledge
graph embedding (KGE) to handle a huge number of entities. However, the
performance of KGE degrades without hyperparameters such as the margin term and
number of negative samples in NS loss being appropriately selected. Currently,
empirical hyperparameter tuning addresses this problem at the cost of
computational time. To solve this problem, we theoretically analyzed NS loss to
assist hyperparameter tuning and understand the better use of the NS loss in
KGE learning. Our theoretical analysis showed that scoring methods with
restricted value ranges, such as TransE and RotatE, require appropriate
adjustment of the margin term or the number of negative samples different from
those without restricted value ranges, such as RESCAL, ComplEx, and DistMult.
We also propose subsampling methods specialized for the NS loss in KGE studied
from a theoretical aspect. Our empirical analysis on the FB15k-237, WN18RR, and
YAGO3-10 datasets showed that the results of actually trained models agree with
our theoretical findings.