An Augmented Transformer Architecture for Natural Language Generation
  Tasks
  The Transformer based neural networks have been showing significant
advantages on most evaluations of various natural language processing and other
sequence-to-sequence tasks due to its inherent architecture based
superiorities. Although the main architecture of the Transformer has been
continuously being explored, little attention was paid to the positional
encoding module. In this paper, we enhance the sinusoidal positional encoding
algorithm by maximizing the variances between encoded consecutive positions to
obtain additional promotion. Furthermore, we propose an augmented Transformer
architecture encoded with additional linguistic knowledge, such as the
Part-of-Speech (POS) tagging, to boost the performance on some natural language
generation tasks, e.g., the automatic translation and summarization tasks.
Experiments show that the proposed architecture attains constantly superior
results compared to the vanilla Transformer.
