Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs
  to be Forgotten
  As the use of machine learning (ML) models is becoming increasingly popular
in many real-world applications, there are practical challenges that need to be
addressed for model maintenance. One such challenge is to 'undo' the effect of
a specific subset of dataset used for training a model. This specific subset
may contain malicious or adversarial data injected by an attacker, which
affects the model performance. Another reason may be the need for a service
provider to remove data pertaining to a specific user to respect the user's
privacy. In both cases, the problem is to 'unlearn' a specific subset of the
training data from a trained model without incurring the costly procedure of
retraining the whole model from scratch. Towards this goal, this paper presents
a Markov chain Monte Carlo-based machine unlearning (MCU) algorithm. MCU helps
to effectively and efficiently unlearn a trained model from subsets of training
dataset. Furthermore, we show that with MCU, we are able to explain the effect
of a subset of a training dataset on the model prediction. Thus, MCU is useful
for examining subsets of data to identify the adversarial data to be removed.
Similarly, MCU can be used to erase the lineage of a user's personal data from
trained ML models, thus upholding a user's "right to be forgotten". We
empirically evaluate the performance of our proposed MCU algorithm on
real-world phishing and diabetes datasets. Results show that MCU can achieve a
desirable performance by efficiently removing the effect of a subset of
training dataset and outperform an existing algorithm that utilizes the
remaining dataset.
