Why KDAC? A general activation function for knowledge discovery
  Deep learning oriented named entity recognition (DNER) has gradually become
the paradigm of knowledge discovery, which greatly promotes domain
intelligence. However, the current activation function of DNER fails to treat
gradient vanishing, no negative output or non-differentiable existence, which
may impede knowledge exploration caused by the omission and incomplete
representation of latent semantics. To break through the dilemma, we present a
novel activation function termed KDAC. Detailly, KDAC is an aggregation
function with multiple conversion modes. The backbone of the activation region
is the interaction between exponent and linearity, and the both ends extend
through adaptive linear divergence, which surmounts the obstacle of gradient
vanishing and no negative output. Crucially, the non-differentiable points are
alerted and eliminated by an approximate smoothing algorithm. KDAC has a series
of brilliant properties, including nonlinear, stable near-linear transformation
and derivative, as well as dynamic style, etc. We perform experiments based on
BERT-BiLSTM-CNN-CRF model on six benchmark datasets containing different domain
knowledge, such as Weibo, Clinical, E-commerce, Resume, HAZOP and People's
daily. The evaluation results show that KDAC is advanced and effective, and can
provide more generalized activation to stimulate the performance of DNER. We
hope that KDAC can be exploited as a promising activation function to devote
itself to the construction of knowledge.
