InferNet for Delayed Reinforcement Tasks: Addressing the Temporal Credit
  Assignment Problem
  The temporal Credit Assignment Problem (CAP) is a well-known and challenging
task in AI. While Reinforcement Learning (RL), especially Deep RL, works well
when immediate rewards are available, it can fail when only delayed rewards are
available or when the reward function is noisy. In this work, we propose
delegating the CAP to a Neural Network-based algorithm named InferNet that
explicitly learns to infer the immediate rewards from the delayed rewards. The
effectiveness of InferNet was evaluated on two online RL tasks: a simple
GridWorld and 40 Atari games; and two offline RL tasks: GridWorld and a
real-life Sepsis treatment task. For all tasks, the effectiveness of using the
InferNet inferred rewards is compared against the immediate and the delayed
rewards with two settings: with noisy rewards and without noise. Overall, our
results show that the effectiveness of InferNet is robust against noisy reward
functions and is an effective add-on mechanism for solving temporal CAP in a
wide range of RL tasks, from classic RL simulation environments to a real-world
RL problem and for both online and offline learning.
