Overview
Imagine a classifier learned from a picture that should now be removed, or from a label that was wrong. Updating the model to make a different prediction on that picture is easy. Keeping the update from damaging other predictions is harder, especially when the rest of the original training set is unavailable. The authors test an edit that uses only the selected examples and the pretrained classifier.
Key Findings
Take one example out of a crowded memory
An image classifier shares its weights across many examples. Forcing it to stop predicting one selected image’s old label can move the decision boundary for other images too. The authors focus on edits where the selected images are available and the rest of the original training data may be difficult to recover. They study two goals. Misclassification moves a selected image away from its original label. Correction gives it a specified replacement label.
Borrow training signals from the examples being removed
The first safeguard makes small targeted perturbations of the selected images. These adversarial examples induce the old classifier to respond as if it had seen other classes. Training on those perturbed examples with their target labels gives the model a signal about boundaries that the edit should preserve. The second safeguard uses sensitivity of the model’s outputs to estimate which weights matter for the selected images. It penalizes changing weights that contributed less to predicting those images.
In the first variant, denotes the selected images and the perturbed copies with target labels. The total loss adds an ordinary classification loss on the copies to the edit objective. Equation 6
Changing the target label is the easy part
| ImageNet-1K model after selecting 64 images for misclassification | Accuracy on selected images ↓ | Accuracy on retained training examples ↑ | Test accuracy ↑ |
|---|---|---|---|
| Before editing | 84.69 | 87.42 | 76.01 |
| Oracle using retained data | 0.00 | 77.81 | 68.08 |
| Negative-gradient update | 0.00 | 35.66 | 31.10 |
| Adversarial regularization | 0.00 | 73.60 | 65.51 |
| Adversarial and importance regularization | 0.00 | 79.03 | 68.43 |
The selected accuracies show that every edit in these rows can change the predictions on the target images. The consequential difference is how much useful behavior survives. The combined regularizer preserves more test accuracy than the negative-gradient baseline but still gives up 7.58 points from the original model. The oracle is a reference with access to retained training data. Table 1
A stronger safeguard can get in the way of a correction
The continual experiment removes CIFAR-100 examples eight at a time. At 128 removals, test accuracy is 36.87 with both regularizers, against 31.11 with adversarial examples alone and 0.98 with the negative-gradient baseline. The same safeguard behaves differently in correction. For 64 natural adversarial ImageNet-A images, adversarial regularization alone corrects 95.31% of the selected images. Adding the importance constraint drops that to 10.94%. A constraint that preserves the old model through many removals can also keep it from learning replacement labels. The paper inspects the resulting labels, feature-space plots, and layer-wise representation changes to examine what the edits preserve. Tables 3–4 · Figures 2–4
Questions & Answers
Q: Why can changing one label break other predictions?
A: The same weights contribute to many examples and classes. An unrestricted update can move the representation for the selected example in a way that also alters useful behavior elsewhere.
Q: Why use perturbed images as a preservation signal?
A: The perturbations are chosen with the pretrained model to elicit other class responses. They provide a way to regularize the edit even when the original retained images cannot be accessed.
Q: Does zero accuracy on selected images prove they were erased?
A: It shows that the classifier's predictions changed under the tested objective. It leaves open whether the model contains signals an attacker could use to infer membership or recover characteristics of those examples.
Q: What happens when an editor tries to change a much larger batch?
A: Larger edits can damage the predictions the model is supposed to preserve. In the reported CIFAR-10 run with 256 selected images, the combined regularizer leaves 45.44% test accuracy. The analysis also finds that representation changes are larger in later layers. A batch procedure should set a retained-performance target and check it after each edit, alongside the targeted changes.
Future Ideas
Treat label correction as a versioned model change
Set up a testbed of image-classifier corrections with independently verified replacement labels. Compare the paper's regularized edit with an unrestricted update and a retrained reference. Check each correction on the selected image, nearby images with the same class, and a frozen holdout set. Keep the results alongside the model version so batches of edits can be rolled back. Start larger correction batches with the adversarial regularizer alone. In Table 4, it corrects 95.31% of a batch of 64 ImageNet-A images, compared with 10.94% using both regularizers.
Measure correction success, accuracy loss, and changes in confidence after each successive batch. For a separate removal evaluation, measure attack success at a fixed query budget and compare to a model trained without the selected records. This separates a useful repair from a claim about deletion.
Sources
Original abstract
Since the recent advent of regulations for data protection (e.g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sen- sitive data in pre-trained models without retraining from scratch. The inherent vulnerability of neural networks to- wards adversarial attacks and unfairness also calls for a robust method to remove or correct information in an instance-wise fashion, while retaining the predictive performance across remaining data. To this end, we consider instance-wise un- learning, of which the goal is to delete information on a set of instances from a pre-trained model, by either misclassi- fying each instance away from its original prediction or re- labeling the instance to a different label. We also propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information. Both methods only require the pre- trained model and data instances to forget, allowing painless application to real-life settings where the entire training set is unavailable. Through extensive experimentation on various image classification benchmarks, we show that our approach effectively preserves knowledge of remaining data while un- learning given instances in both single-task and continual un- learning scenarios.