Feature Shift Detection: Localizing Which Features Have Shifted via
  Conditional Distribution Tests
  While previous distribution shift detection approaches can identify if a
shift has occurred, these approaches cannot localize which specific features
have caused a distribution shift -- a critical step in diagnosing or fixing any
underlying issue. For example, in military sensor networks, users will want to
detect when one or more of the sensors has been compromised, and critically,
they will want to know which specific sensors might be compromised. Thus, we
first define a formalization of this problem as multiple conditional
distribution hypothesis tests and propose both non-parametric and parametric
statistical tests. For both efficiency and flexibility, we then propose to use
a test statistic based on the density model score function (i.e. gradient with
respect to the input) -- which can easily compute test statistics for all
dimensions in a single forward and backward pass. Any density model could be
used for computing the necessary statistics including deep density models such
as normalizing flows or autoregressive models. We additionally develop methods
for identifying when and where a shift occurs in multivariate time-series data
and show results for multiple scenarios using realistic attack models on both
simulated and real world data.
