MCRM: Mother Compact Recurrent Memory
  LSTMs and GRUs are the most common recurrent neural network architectures
used to solve temporal sequence problems. The two architectures have differing
data flows dealing with a common component called the cell state (also referred
to as the memory). We attempt to enhance the memory by presenting a
modification that we call the Mother Compact Recurrent Memory (MCRM). MCRMs are
a type of a nested LSTM-GRU architecture where the cell state is the GRU hidden
state. The concatenation of the forget gate and input gate interactions from
the LSTM are considered an input to the GRU cell. Because MCRMs has this type
of nesting, MCRMs have a compact memory pattern consisting of neurons that acts
explicitly in both long-term and short-term fashions. For some specific tasks,
empirical results show that MCRMs outperform previously used architectures.
