On the rate of convergence of a classifier based on a Transformer
  encoder
  Pattern recognition based on a high-dimensional predictor is considered. A
classifier is defined which is based on a Transformer encoder. The rate of
convergence of the misclassification probability of the classifier towards the
optimal misclassification probability is analyzed. It is shown that this
classifier is able to circumvent the curse of dimensionality provided the
aposteriori probability satisfies a suitable hierarchical composition model.
Furthermore, the difference between Transformer classifiers analyzed
theoretically in this paper and Transformer classifiers used nowadays in
practice are illustrated by considering classification problems in natural
language processing.
