Overview
Trying to make a model forget a memorized passage can make unrelated text worse. Simply pushing the probability of an unwanted token down may disturb the distribution the model uses everywhere else. This study tests losses and adapter initializations that aim to suppress a target more selectively.
Key Findings
During training, a model can learn to finish a passage with the very text a user later asks it to forget. An obvious edit is to reduce the probability of that continuation. This paper follows two consequences of that edit. What happens to the alternatives at that position, and how much of the model has to change?
Give the model a particular alternative
Ordinary gradient ascent pushes down the probability of the target token by pushing up the scores of every other token. Its objective can also keep improving as the target probability goes toward zero. The paper's inverted hinge loss sets a bounded goal and gives special weight to the strongest alternative. Let be the probability of the target token in the given context and the highest probability of any other token. The per-token term is
Minimizing this term moves probability away from the target and toward a concrete alternative. The authors train with an auxiliary retain set as well, so the update still receives a signal to keep predicting ordinary text. Loss and update, Sections 3.3 and 3.5
Start the adapters where the forget set has more influence
LoRA edits a large weight matrix with a product of two small matrices while the original weights stay frozen. That lowers the number of trainable parameters. But in the preliminary tests, simply using standard LoRA could take longer or damage general language performance by the time it suppressed the targeted text. FLoRA uses the squared gradients of the language-model loss to estimate parameter importance, comparing the forget set with the retain set. It uses those weights to initialize the low-rank components. The paper aggregates importance by row and obtains the initialization through a singular value decomposition of the rescaled matrix, avoiding a separate iterative fitting procedure. Adapter initialization, Section 3.4
Check both extraction and the abilities that survive
The team evaluated three GPT-Neo sizes on memorized text from the Training Data Extraction Challenge. The stopping rule compares two measures against a held-out set: overlap with the target text in ten-token sequences and accuracy when predicting its next tokens. General reasoning, dialogue and language-modeling tests separately track what survives the update.
| GPT-Neo 2.7B, Table 1 | Ten-token extraction likelihood (%) | Memorization accuracy (%) | Reasoning accuracy (%) | Pile perplexity | Epochs |
|---|---|---|---|---|---|
| Before editing | 70.4 | 93.4 | 52.3 | 10.4 | — |
| LoRA | 0.1 | 20.4 | 45.9 | 61.1 | 14.0 |
| LoRA + inverted hinge loss | 0.0 | 26.7 | 49.6 | 22.2 | 17.8 |
| FLoRA + inverted hinge loss | 0.1 | 28.5 | 49.6 | 16.0 | 10.3 |
With the hinge loss held fixed, FLoRA reaches the criterion in fewer epochs and leaves perplexity closer to the original model's value. Plain LoRA leaves a lower memorization accuracy on this test, but the accompanying perplexity increase shows the cost of optimizing for forgetting alone. The apparent 0.0% is a rounded measure under the paper's extraction protocol. Probing under different prompts or sampling rules would be a separate experiment. GPT-Neo results and evaluation protocol, Table 1 and Section 4
Questions & Answers
The opening answers are simulated author responses based on the paper.
Q: Why use a low-rank adapter?
A: It limits how many parameters are trainable. The initialization determines where the resulting update begins.
Q: Does low extraction prove the data are erased?
A: It measures what the evaluation can recover. A different attack may still elicit the content, and other forgetting metrics can remain nonzero.
Q: What counts toward the reported efficiency of adapter unlearning?
A: The final comparison combines the inverted hinge loss and FLoRA. How much time the method saves depends in part on when the experiment stops. An end-to-end comparison also needs to count the computation required to estimate Fisher information for initialization. That separates a faster optimization phase from a lower total cost.
Future Ideas
Audit a synthetic deletion request before retraining
Plant fictional records and related decoys in a small model’s training data. Compare a full retrain, ordinary LoRA editing, and the proposed adapter variants. Score extraction from exact and paraphrased prompts at several attempt budgets, plus unrelated and neighboring facts. Compare the protocol with the paper’s cited TOFU benchmark and count Fisher estimation on the forget and retain sets in the total editing cost. Keep the planted records synthetic to avoid reproducing private data.
Count Fisher estimation in the total compute and compare with a reference trained without the planted targets. Probe related records as well as the exact target.