Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning
  Advances in artificial intelligence often stem from the development of new
environments that abstract real-world situations into a form where research can
be done conveniently. This paper contributes such an environment based on ideas
inspired by elementary Microeconomics. Agents learn to produce resources in a
spatially complex world, trade them with one another, and consume those that
they prefer. We show that the emergent production, consumption, and pricing
behaviors respond to environmental conditions in the directions predicted by
supply and demand shifts in Microeconomics. We also demonstrate settings where
the agents' emergent prices for goods vary over space, reflecting the local
abundance of goods. After the price disparities emerge, some agents then
discover a niche of transporting goods between regions with different
prevailing prices -- a profitable strategy because they can buy goods where
they are cheap and sell them where they are expensive. Finally, in a series of
ablation experiments, we investigate how choices in the environmental rewards,
bartering actions, agent architecture, and ability to consume tradable goods
can either aid or inhibit the emergence of this economic behavior. This work is
part of the environment development branch of a research program that aims to
build human-like artificial general intelligence through multi-agent
interactions in simulated societies. By exploring which environment features
are needed for the basic phenomena of elementary microeconomics to emerge
automatically from learning, we arrive at an environment that differs from
those studied in prior multi-agent reinforcement learning work along several
dimensions. For example, the model incorporates heterogeneous tastes and
physical abilities, and agents negotiate with one another as a grounded form of
communication.
