Using Cyber Terrain in Reinforcement Learning for Penetration Testing
  Reinforcement learning (RL) has been applied to attack graphs for penetration
testing, however, trained agents do not reflect reality because the attack
graphs lack operational nuances typically captured within the intelligence
preparation of the battlefield (IPB) that include notions of (cyber) terrain.
In particular, current practice constructs attack graphs exclusively using the
Common Vulnerability Scoring System (CVSS) and its components. We present
methods for constructing attack graphs using notions from IPB on cyber terrain
analysis of obstacles, avenues of approach, key terrain, observation and fields
of fire, and cover and concealment. We demonstrate our methods on an example
where firewalls are treated as obstacles and represented in (1) the reward
space and (2) the state dynamics. We show that terrain analysis can be used to
bring realism to attack graphs for RL.
