Reinforcement Learning for Feedback-Enabled Cyber Resilience
  Digitization and remote connectivity have enlarged the attack surface and
made cyber systems more vulnerable. As attackers become increasingly
sophisticated and resourceful, mere reliance on traditional cyber protection,
such as intrusion detection, firewalls, and encryption, is insufficient to
secure the cyber systems. Cyber resilience provides a new security paradigm
that complements inadequate protection with resilience mechanisms. A
Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and
uncertainties in real-time and strategically responds to them to maintain
critical functions of the cyber systems in the event of successful attacks.
Feedback architectures play a pivotal role in enabling the online sensing,
reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an
essential tool that epitomizes the feedback architectures for cyber resilience.
It allows the CRM to provide sequential responses to attacks with limited or
without prior knowledge of the environment and the attacker. In this work, we
review the literature on RL for cyber resilience and discuss cyber resilience
against three major types of vulnerabilities, i.e., posture-related,
information-related, and human-related vulnerabilities. We introduce three
application domains of CRMs: moving target defense, defensive cyber deception,
and assistive human security technologies. The RL algorithms also have
vulnerabilities themselves. We explain the three vulnerabilities of RL and
present attack models where the attacker targets the information exchanged
between the environment and the agent: the rewards, the state observations, and
the action commands. We show that the attacker can trick the RL agent into
learning a nefarious policy with minimum attacking effort. Lastly, we discuss
the future challenges of RL for cyber security and resilience and emerging
applications of RL-based CRMs.
