Adversarial Representation Learning With Closed-Form Solvers
  Adversarial representation learning aims to learn data representations for a
target task while removing unwanted sensitive information at the same time.
Existing methods learn model parameters iteratively through stochastic gradient
descent-ascent, which is often unstable and unreliable in practice. To overcome
this challenge, we adopt closed-form solvers for the adversary and target task.
We model them as kernel ridge regressors and analytically determine an
upper-bound on the optimal dimensionality of representation. Our solution,
dubbed OptNet-ARL, reduces to a stable one one-shot optimization problem that
can be solved reliably and efficiently. OptNet-ARL can be easily generalized to
the case of multiple target tasks and sensitive attributes. Numerical
experiments, on both small and large scale datasets, show that, from an
optimization perspective, OptNet-ARL is stable and exhibits three to five times
faster convergence. Performance wise, when the target and sensitive attributes
are dependent, OptNet-ARL learns representations that offer a better trade-off
front between (a) utility and bias for fair classification and (b) utility and
privacy by mitigating leakage of private information than existing solutions.
