RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN
  Radio access network (RAN) technologies continue to witness massive growth,
with Open RAN gaining the most recent momentum. In the O-RAN specifications,
the RAN intelligent controller (RIC) serves as an automation host. This article
introduces principles for machine learning (ML), in particular, reinforcement
learning (RL) relevant for the O-RAN stack. Furthermore, we review
state-of-the-art research in wireless networks and cast it onto the RAN
framework and the hierarchy of the O-RAN architecture. We provide a taxonomy of
the challenges faced by ML/RL models throughout the development life-cycle:
from the system specification to production deployment (data acquisition, model
design, testing and management, etc.). To address the challenges, we integrate
a set of existing MLOps principles with unique characteristics when RL agents
are considered. This paper discusses a systematic life-cycle model development,
testing and validation pipeline, termed: RLOps. We discuss all fundamental
parts of RLOps, which include: model specification, development and
distillation, production environment serving, operations monitoring,
safety/security and data engineering platform. Based on these principles, we
propose the best practices for RLOps to achieve an automated and reproducible
model development process.
