From Crystallized Adaptivity to Fluid Adaptivity in Deep Reinforcement
  Learning -- Insights from Biological Systems on Adaptive Flexibility
  Recent developments in machine-learning algorithms have led to impressive
performance increases in many traditional application scenarios of artificial
intelligence research. In the area of deep reinforcement learning, deep
learning functional architectures are combined with incremental learning
schemes for sequential tasks that include interaction-based, but often delayed
feedback. Despite their impressive successes, modern machine-learning
approaches, including deep reinforcement learning, still perform weakly when
compared to flexibly adaptive biological systems in certain naturally occurring
scenarios. Such scenarios include transfers to environments different than the
ones in which the training took place or environments that dynamically change,
both of which are often mastered by biological systems through a capability
that we here term "fluid adaptivity" to contrast it from the much slower
adaptivity ("crystallized adaptivity") of the prior learning from which the
behavior emerged. In this article, we derive and discuss research strategies,
based on analyzes of fluid adaptivity in biological systems and its neuronal
modeling, that might aid in equipping future artificially intelligent systems
with capabilities of fluid adaptivity more similar to those seen in some
biologically intelligent systems. A key component of this research strategy is
the dynamization of the problem space itself and the implementation of this
dynamization by suitably designed flexibly interacting modules.
