Privacy-Preserving Reinforcement Learning Beyond Expectation
  Cyber and cyber-physical systems equipped with machine learning algorithms
such as autonomous cars share environments with humans. In such a setting, it
is important to align system (or agent) behaviors with the preferences of one
or more human users. We consider the case when an agent has to learn behaviors
in an unknown environment. Our goal is to capture two defining characteristics
of humans: i) a tendency to assess and quantify risk, and ii) a desire to keep
decision making hidden from external parties. We incorporate cumulative
prospect theory (CPT) into the objective of a reinforcement learning (RL)
problem for the former. For the latter, we use differential privacy. We design
an algorithm to enable an RL agent to learn policies to maximize a CPT-based
objective in a privacy-preserving manner and establish guarantees on the
privacy of value functions learned by the algorithm when rewards are
sufficiently close. This is accomplished through adding a calibrated noise
using a Gaussian process mechanism at each step. Through empirical evaluations,
we highlight a privacy-utility tradeoff and demonstrate that the RL agent is
able to learn behaviors that are aligned with that of a human user in the same
environment in a privacy-preserving manner
