Stability Analysis of Recurrent Neural Networks by IQC with Copositive
  Mutipliers
  This paper is concerned with the stability analysis of the recurrent neural
networks (RNNs) by means of the integral quadratic constraint (IQC) framework.
The rectified linear unit (ReLU) is typically employed as the activation
function of the RNN, and the ReLU has specific nonnegativity properties
regarding its input and output signals. Therefore, it is effective if we can
derive IQC-based stability conditions with multipliers taking care of such
nonnegativity properties. However, such nonnegativity (linear) properties are
hardly captured by the existing multipliers defined on the positive
semidefinite cone. To get around this difficulty, we loosen the standard
positive semidefinite cone to the copositive cone, and employ copositive
multipliers to capture the nonnegativity properties. We show that, within the
framework of the IQC, we can employ copositive multipliers (or their inner
approximation) together with existing multipliers such as Zames-Falb
multipliers and polytopic bounding multipliers, and this directly enables us to
ensure that the introduction of the copositive multipliers leads to better (no
more conservative) results. We finally illustrate the effectiveness of the
IQC-based stability conditions with the copositive multipliers by numerical
examples.
