Adaptive model selection in photonic reservoir computing by
  reinforcement learning
  Photonic reservoir computing is an emergent technology toward beyond-Neumann
computing. Although photonic reservoir computing provides superior performance
in environments whose characteristics are coincident with the training datasets
for the reservoir, the performance is significantly degraded if these
characteristics deviate from the original knowledge used in the training phase.
Here, we propose a scheme of adaptive model selection in photonic reservoir
computing using reinforcement learning. In this scheme, a temporal waveform is
generated by different dynamic source models that change over time. The system
autonomously identifies the best source model for the task of time series
prediction using photonic reservoir computing and reinforcement learning. We
prepare two types of output weights for the source models, and the system
adaptively selected the correct model using reinforcement learning, where the
prediction errors are associated with rewards. We succeed in adaptive model
selection when the source signal is temporally mixed, having originally been
generated by two different dynamic system models, as well as when the signal is
a mixture from the same model but with different parameter values. This study
paves the way for autonomous behavior in photonic artificial intelligence and
could lead to new applications in load forecasting and multi-objective control,
where frequent environment changes are expected.
