Delayed Q-update: A novel credit assignment technique for deriving an
  optimal operation policy for the Grid-Connected Microgrid
  A microgrid is an innovative system that integrates distributed energy
resources to supply electricity demand within electrical boundaries. This study
proposes an approach for deriving a desirable microgrid operation policy that
enables sophisticated controls in the microgrid system using the proposed novel
credit assignment technique, delayed-Q update. The technique employs novel
features such as the ability to tackle and resolve the delayed effective
property of the microgrid, which prevents learning agents from deriving a
well-fitted policy under sophisticated controls. The proposed technique tracks
the history of the charging period and retroactively assigns an adjusted value
to the ESS charging control. The operation policy derived using the proposed
approach is well-fitted for the real effects of ESS operation because of the
process of the technique. Therefore, it supports the search for a near-optimal
operation policy under a sophisticatedly controlled microgrid environment. To
validate our technique, we simulate the operation policy under a real-world
grid-connected microgrid system and demonstrate the convergence to a
near-optimal policy by comparing performance measures of our policy with
benchmark policy and optimal policy.
