Learning to Bid Long-Term: Multi-Agent Reinforcement Learning with
  Long-Term and Sparse Reward in Repeated Auction Games
  We propose a multi-agent distributed reinforcement learning algorithm that
balances between potentially conflicting short-term reward and sparse, delayed
long-term reward, and learns with partial information in a dynamic environment.
We compare different long-term rewards to incentivize the algorithm to maximize
individual payoff and overall social welfare. We test the algorithm in two
simulated auction games, and demonstrate that 1) our algorithm outperforms two
benchmark algorithms in a direct competition, with cost to social welfare, and
2) our algorithm's aggressive competitive behavior can be guided with the
long-term reward signal to maximize both individual payoff and overall social
welfare.
