Learning to Operate an Electric Vehicle Charging Station Considering
  Vehicle-grid Integration
  The rapid adoption of electric vehicles (EVs) calls for the widespread
installation of EV charging stations. To maximize the profitability of charging
stations, intelligent controllers that provide both charging and electric grid
services are in great need. However, it is challenging to determine the optimal
charging schedule due to the uncertain arrival time and charging demands of
EVs. In this paper, we propose a novel centralized allocation and decentralized
execution (CADE) reinforcement learning (RL) framework to maximize the charging
station's profit. In the centralized allocation process, EVs are allocated to
either the waiting or charging spots. In the decentralized execution process,
each charger makes its own charging/discharging decision while learning the
action-value functions from a shared replay memory. This CADE framework
significantly improves the scalability and sample efficiency of the RL
algorithm. Numerical results show that the proposed CADE framework is both
computationally efficient and scalable, and significantly outperforms the
baseline model predictive control (MPC). We also provide an in-depth analysis
of the learned action-value function to explain the inner working of the
reinforcement learning agent.
