Deep Unlearning via Randomized Conditionally Independent Hessians
  Recent legislation has led to interest in machine unlearning, i.e., removing
specific training samples from a predictive model as if they never existed in
the training dataset. Unlearning may also be required due to
corrupted/adversarial data or simply a user's updated privacy requirement. For
models which require no training (k-NN), simply deleting the closest original
sample can be effective. But this idea is inapplicable to models which learn
richer representations. Recent ideas leveraging optimization-based updates
scale poorly with the model dimension d, due to inverting the Hessian of the
loss function. We use a variant of a new conditional independence coefficient,
L-CODEC, to identify a subset of the model parameters with the most semantic
overlap on an individual sample level. Our approach completely avoids the need
to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we
premise that L-CODEC is also suitable for deep unlearning, as well as other
applications in vision. Compared to alternatives, L-CODEC makes approximate
unlearning possible in settings that would otherwise be infeasible, including
vision models used for face recognition, person re-identification and NLP
models that may require unlearning samples identified for exclusion. Code can
be found at https://github.com/vsingh-group/LCODEC-deep-unlearning/
