Deep Reinforcement Learning for Constrained Field Development
  Optimization in Subsurface Two-phase Flow
  We present a deep reinforcement learning-based artificial intelligence agent
that could provide optimized development plans given a basic description of the
reservoir and rock/fluid properties with minimal computational cost. This
artificial intelligence agent, comprising of a convolutional neural network,
provides a mapping from a given state of the reservoir model, constraints, and
economic condition to the optimal decision (drill/do not drill and well
location) to be taken in the next stage of the defined sequential field
development planning process. The state of the reservoir model is defined using
parameters that appear in the governing equations of the two-phase flow. A
feedback loop training process referred to as deep reinforcement learning is
used to train an artificial intelligence agent with such a capability. The
training entails millions of flow simulations with varying reservoir model
descriptions (structural, rock and fluid properties), operational constraints,
and economic conditions. The parameters that define the reservoir model,
operational constraints, and economic conditions are randomly sampled from a
defined range of applicability. Several algorithmic treatments are introduced
to enhance the training of the artificial intelligence agent. After appropriate
training, the artificial intelligence agent provides an optimized field
development plan instantly for new scenarios within the defined range of
applicability. This approach has advantages over traditional optimization
algorithms (e.g., particle swarm optimization, genetic algorithm) that are
generally used to find a solution for a specific field development scenario and
typically not generalizable to different scenarios.
