CMAX++ : Leveraging Experience in Planning and Execution using
  Inaccurate Models
  Given access to accurate dynamical models, modern planning approaches are
effective in computing feasible and optimal plans for repetitive robotic tasks.
However, it is difficult to model the true dynamics of the real world before
execution, especially for tasks requiring interactions with objects whose
parameters are unknown. A recent planning approach, CMAX, tackles this problem
by adapting the planner online during execution to bias the resulting plans
away from inaccurately modeled regions. CMAX, while being provably guaranteed
to reach the goal, requires strong assumptions on the accuracy of the model
used for planning and fails to improve the quality of the solution over
repetitions of the same task. In this paper we propose CMAX++, an approach that
leverages real-world experience to improve the quality of resulting plans over
successive repetitions of a robotic task. CMAX++ achieves this by integrating
model-free learning using acquired experience with model-based planning using
the potentially inaccurate model. We provide provable guarantees on the
completeness and asymptotic convergence of CMAX++ to the optimal path cost as
the number of repetitions increases. CMAX++ is also shown to outperform
baselines in simulated robotic tasks including 3D mobile robot navigation where
the track friction is incorrectly modeled, and a 7D pick-and-place task where
the mass of the object is unknown leading to discrepancy between true and
modeled dynamics.
