Rayleigh EigenDirections (REDs): GAN latent space traversals for
  multidimensional features
  We present a method for finding paths in a deep generative model's latent
space that can maximally vary one set of image features while holding others
constant. Crucially, unlike past traversal approaches, ours can manipulate
multidimensional features of an image such as facial identity and pixels within
a specified region. Our method is principled and conceptually simple: optimal
traversal directions are chosen by maximizing differential changes to one
feature set such that changes to another set are negligible. We show that this
problem is nearly equivalent to one of Rayleigh quotient maximization, and
provide a closed-form solution to it based on solving a generalized eigenvalue
equation. We use repeated computations of the corresponding optimal directions,
which we call Rayleigh EigenDirections (REDs), to generate appropriately curved
paths in latent space. We empirically evaluate our method using StyleGAN2 on
two image domains: faces and living rooms. We show that our method is capable
of controlling various multidimensional features out of the scope of previous
latent space traversal methods: face identity, spatial frequency bands, pixels
within a region, and the appearance and position of an object. Our work
suggests that a wealth of opportunities lies in the local analysis of the
geometry and semantics of latent spaces.
