Selective Forgetting of Deep Networks at a Finer Level than Samples
  Selective forgetting or removing information from deep neural networks (DNNs)
is essential for continual learning and is challenging in controlling the DNNs.
Such forgetting is crucial also in a practical sense since the deployed DNNs
may be trained on the data with outliers, poisoned by attackers, or with
leaked/sensitive information. In this paper, we formulate selective forgetting
for classification tasks at a finer level than the samples' level. We specify
the finer level based on four datasets distinguished by two conditions: whether
they contain information to be forgotten and whether they are available for the
forgetting procedure. Additionally, we reveal the need for such formulation
with the datasets by showing concrete and practical situations. Moreover, we
introduce the forgetting procedure as an optimization problem on three
criteria; the forgetting, the correction, and the remembering term.
Experimental results show that the proposed methods can make the model forget
to use specific information for classification. Notably, in specific cases, our
methods improved the model's accuracy on the datasets, which contains
information to be forgotten but is unavailable in the forgetting procedure.
Such data are unexpectedly found and misclassified in actual situations.
