DSDF: An approach to handle stochastic agents in collaborative
  multi-agent reinforcement learning
  Multi-Agent reinforcement learning has received lot of attention in recent
years and have applications in many different areas. Existing methods involving
Centralized Training and Decentralized execution, attempts to train the agents
towards learning a pattern of coordinated actions to arrive at optimal joint
policy. However if some agents are stochastic to varying degrees of
stochasticity, the above methods often fail to converge and provides poor
coordination among agents. In this paper we show how this stochasticity of
agents, which could be a result of malfunction or aging of robots, can add to
the uncertainty in coordination and there contribute to unsatisfactory global
coordination. In this case, the deterministic agents have to understand the
behavior and limitations of the stochastic agents while arriving at optimal
joint policy. Our solution, DSDF which tunes the discounted factor for the
agents according to uncertainty and use the values to update the utility
networks of individual agents. DSDF also helps in imparting an extent of
reliability in coordination thereby granting stochastic agents tasks which are
immediate and of shorter trajectory with deterministic ones taking the tasks
which involve longer planning. Such an method enables joint co-ordinations of
agents some of which may be partially performing and thereby can reduce or
delay the investment of agent/robot replacement in many circumstances. Results
on benchmark environment for different scenarios shows the efficacy of the
proposed approach when compared with existing approaches.
