Overview
When a language model memorizes a passage, one way to suppress it is to train the model away from its own original prediction. Push too hard and the update can damage unrelated language skills. LoKU changes both the push and where it acts. Its loss reduces the preference for a targeted token, while its initialization helps a small adapter focus on weights associated with the material to forget.
Key Findings
A publisher might ask a model provider to remove a passage it can recite. The provider wants that passage to become harder to elicit while keeping the model able to answer ordinary questions. An update that simply makes the model worse at generating text can look successful on the removal test. The researchers give both sides of the request their own measurements.
Push a memorized answer below alternatives
The study starts with separate forget and retain sets. The inverted hinge loss compares the probability of a target token from the forget set with the best alternative probability. For target token and prefix , the loss at one position is
Minimizing it lowers the target relative to its most likely competitor. Its range is bounded, and the target and competitor receive the most direct gradient. The expression continues to vary when the competitor is already ahead. Section 3.3
LoKU uses low-rank adaptation, or LoRA, to train only a small part of the model. Its initializer, FILA, compares empirical Fisher information computed on the forget set and the retain set. This uses the sensitivity of model predictions to weights as a guide. A weighted low-rank approximation initializes the adapter toward weights more associated with the forget data. The authors combine that initialization with the inverted hinge loss.
Compare forgetting and retained capability together
The following rows are from Table 1 for GPT-Neo 125M. Each forget set has 32 sequences of 200 tokens, and the LoRA rows use rank 16. EL10 measures extractability using a continuation-based score and MA is memorization accuracy. Reasoning and dialogue summarize separate downstream tasks. Low perplexity indicates better prediction on the Pile validation set.
| Method | EL10 ↓ | MA ↓ | Reasoning ↑ | Dialogue F1 ↑ | Pile perplexity ↓ |
|---|---|---|---|---|---|
| Before unlearning | 30.9 | 77.4 | 43.4 | 9.4 | 17.8 |
| Full-parameter gradient difference | 0.7 | 24.9 | 42.4 | 5.9 | 54.2 |
| LoRA gradient difference | 0.3 | 20.6 | 40.8 | 2.5 | 129.4 |
| LoKU | 0.3 | 23.9 | 42.2 | 10.1 | 24.0 |
LoKU trains 1.6% of the parameters in this setup. Its advantage over the vanilla adapter is especially visible on dialogue and perplexity. Its reasoning score and perplexity still differ from the original model. Other GPT-Neo sizes produce different trade-offs. The authors also report that FILA can over-unlearn on TOFU, where choosing when to stop depends on a reference model.
Pick where to make the change
Low-rank adapters restrict how many new parameters can move. That makes the choice of starting directions matter. FILA looks for directions that respond strongly to examples earmarked for forgetting relative to those the model should retain. The paper tests this initialization as well as the loss. This separates the question of where the model is changed from the question of which outputs the loss pushes down.
Compare against a model that never saw the record
The second set of experiments uses TOFU, which contains questions about fictitious authors. A reference model trained only on the retained profiles offers a comparison for what forgetting could look like. Section 4.3 measures how similar the model's outputs are to that reference and how much utility the model preserves. Its synthetic design makes the comparison possible. A request to remove facts from a real model could involve multiple redundant sources and would need a wider test than the training examples alone.
Questions & Answers
The opening answers are simulated author responses based on the paper.
Q: Is fewer trainable parameters the same as lower total cost?
A: The adapter reduces the number of trainable weights. Preparing its initialization also costs compute for Fisher estimates and matrix decompositions. The unlearning FLOP comparison does not break that cost out separately, so a service should measure it.
Q: Why compare sensitivities for two different sets?
A: Weights useful for the target text may also support ordinary language generation. Comparing their sensitivities gives us a way to prioritize weights more associated with the forget set while trying to preserve the retain set.
Q: How would you test if an unusual prompt can recover the passage?
A: Use a held-out set of direct prefixes, paraphrases, and adaptive extraction attempts. Keep the attacker’s budget fixed across models and check for exact text as well as retained facts that can still identify the target.
Q: Can forgetting too much make the model look better on extraction tests?
A: Yes. A model that can no longer answer a broader class of questions may also be harder to prompt for the target. The TOFU results illustrate why the stopping rule matters. Test alternative prompts and unrelated capabilities, and compare with retraining without the target when feasible. A lower extraction score alone cannot establish permanent removal.
Future Ideas
An unlearning trial with neighboring records
Build a synthetic corpus with fictional entities, including near-neighbor names and facts. Train a small model, remove one entity at a time, and run LoKU alongside full fine-tuning, ordinary LoRA, and a model retrained without the entity.
Measure targeted recall and extraction, performance on neighboring facts, and recovery after an unrelated follow-on fine-tune. Separate the prompts used to guide the edit from those used to evaluate it. Compare utility at matched levels of targeted extraction and log the initialization and update costs separately.
Use the retrained model as a reference for choosing a stopping rule on development records. Evaluate whether practical held-out extraction and retained-utility measures recover those decisions when a reference model is unavailable.
Sources
Original abstract
Large Language Models (LLMs) have demonstrated strong reasoning and memorization capabilities via pretraining on massive textual corpora. However, this poses risk of privacy and copyright violations, highlighting the need for efficient machine unlearning methods that remove sensitive data without retraining from scratch. While Gradient Ascent (GA) is commonly used to unlearn by reducing the likelihood of generating unwanted content, it leads to unstable optimization and catastrophic forgetting of retrained knowledge. We find that combining GA with low-rank adaptation results in poor trade-offs between computational cost and generative performance. To address these challenges, we propose Low-rank Knowledge Unlearning (LoKU), a novel framework that enables robust and efficient unlearning for LLMs. First, we introduce Inverted Hinge Loss, which suppresses unwanted tokens while maintaining fluency by boosting the probability of the next most likely token. Second, we develop a data-adaptive initialization for LoRA adapters via low-rank approximation weighted with relative Fisher information, thereby focusing updates on parameters critical for removing targeted knowledge. Experiments on the Training Data Extraction Challenge dataset using GPT-Neo models as well as on the TOFU benchmark with Phi-1.5B and Llama2-7B models demonstrate that our approach effectively removes sensitive information while maintaining reasoning and generative capabilities with minimal impact. Our implementation can be found in https://github.com/csm9493/efficient-llm-unlearning.