C-Learning: Horizon-Aware Cumulative Accessibility Estimation
  Multi-goal reaching is an important problem in reinforcement learning needed
to achieve algorithmic generalization. Despite recent advances in this field,
current algorithms suffer from three major challenges: high sample complexity,
learning only a single way of reaching the goals, and difficulties in solving
complex motion planning tasks. In order to address these limitations, we
introduce the concept of cumulative accessibility functions, which measure the
reachability of a goal from a given state within a specified horizon. We show
that these functions obey a recurrence relation, which enables learning from
offline interactions. We also prove that optimal cumulative accessibility
functions are monotonic in the planning horizon. Additionally, our method can
trade off speed and reliability in goal-reaching by suggesting multiple paths
to a single goal depending on the provided horizon. We evaluate our approach on
a set of multi-goal discrete and continuous control tasks. We show that our
method outperforms state-of-the-art goal-reaching algorithms in success rate,
sample complexity, and path optimality. Our code is available at
https://github.com/layer6ai-labs/CAE, and additional visualizations can be
found at https://sites.google.com/view/learning-cae/.
