Overview
A small assistant can be fast enough to run on a device, yet adding specialized knowledge may weaken skills it already has. A polished answer from a much stronger model may be a poor lesson if it follows patterns the smaller model rarely generates. SynthSelf tests a different teacher. It asks a slightly larger model from the same family to rewrite the example, then checks whether the extracted final answer matches the reference before training on the rewrite.
Key Findings
A lesson can teach its answer and accidentally change the student
A small language model can learn to call a new tool or answer specialized science questions. After training, it may also do worse on coding or general instruction-following. SynthSelf treats the new examples as lessons with two ingredients. One is the knowledge the student needs. The other is the form in which the examples present it. The authors ask whether a familiar form can carry a new lesson with less disruption to what the model already does.
Follow the movement in what the model tends to write
The first diagnostic trains Qwen2.5-3B-Instruct on 23 size-matched datasets and compares how much its output distribution changes with how much performance it loses on five non-math benchmarks. The plot in Figure 2 reports a negative correlation of between generation shift and average retention change. The authors then take the same math dataset and make its demonstrations more familiar to the student while retaining their task content. In that controlled comparison, the lower-shift training loses less performance on HumanEval and performs better on MATH500 than training with the original higher-shift form.
Self-distillation tries to make the demonstrations sound familiar by having the original model rewrite them. The weakness is especially visible at small sizes. Figure 3 shows that the smallest models are more likely to produce wrong training targets or drop the worked reasoning when asked to rewrite an example. A student can stay close to its own style while supplying itself with a weaker lesson.
Find a stronger tutor who still speaks the student’s language
SynthSelf uses a larger model to rewrite each gold demonstration offline. The tutor sees the original question and its answer, so it can preserve the lesson while restating the reasoning. An answer extractor discards rewrites that fail to preserve the gold result. The researchers prefer tutors whose output remains probable under the student. Equation 4 measures that fit with the negative log-likelihood of an accepted rewrite under the original policy .
Here is the question and is the part of the rewritten answer before token . A lower value means the student assigns more probability to the tutor’s wording. Same-family tutors provide more familiar examples in the tested Qwen2.5 setting. The authors compare this with ordinary distillation, where the tutor receives only the question and must supply its own answer.
A stronger tutor can produce a weaker student
Table 2 compares tutors on chemistry and tool use, averaging results over the three student sizes. A stronger model may produce more correct tool-use examples and still train a student that acquires less from them. The largest tutors also cost more to run.
| Rewrite tutor, Table 2 | Science acquisition | Tool-use acquisition | Tool-use rewrite accuracy | Tool-use generation PFLOPs |
|---|---|---|---|---|
| Qwen2.5-7B | 66.8 | 55.7 | 84.1% | 39.0 |
| Qwen2.5-14B | 64.1 | 60.1 | 81.8% | 76.0 |
| Qwen3.5-27B | 65.1 | 50.5 | 95.8% | 142.1 |
| Gemma-4-31B | 60.8 | 56.0 | 99.3% | 167.9 |
Across the four acquisition evaluations, Table 1 gives both SynthSelf variants 56.4 on average versus 34.2 for the base and 52.2 for standard supervised fine-tuning. Retention scores are 41.7 and 41.8, compared with 42.8 for the base. Figure 1 puts those two outcomes on the same plot. The method learns substantially more on the new tasks in these experiments and still gives up some earlier performance.
The rejection-sampling ablation shows that filtering helps both SynthSelf and ordinary off-policy distillation. Comparing the same tutor with and without access to the demonstration helps isolate where the knowledge comes from. The medical setting shows little advantage from giving the tutor the demonstration, which the authors suggest may reflect tutors already knowing the answers. The broader experiment is strongest for domains with checkable outputs and students whose response distribution can be estimated.
Questions & Answers
The opening answers are simulated author responses based on the paper.
Q: Why not use the most powerful teacher available?
A: Our experiments suggest that the answer must also fit the student’s distribution. A stronger teacher may produce useful supervision, but the student may learn it less reliably when its response patterns differ substantially.
Q: Why generate rewrites before training?
A: Offline generation lets us apply the final-answer check and reject failed rewrites before they reach the student. This separates filtering from the optimization step.
Q: Does the method eliminate forgetting?
A: Our results report competitive retention among training methods. The unchanged base model still has a higher overall retention average. The method trades off learning new material and preserving old capabilities.
Q: What would establish that model-family compatibility is doing the work?
A: A same-family aligner may share vocabulary, response conventions, and training history with the student. The reported comparisons cannot separate those possibilities. A more diagnostic test would compare accepted examples of similar difficulty and use matched acceptance rates, training tokens, and several independent updates.
Future Ideas
Update a field assistant after a manual changes
Start with a small assistant that answers questions about equipment and maintenance procedures. When a manual is revised, use a larger sibling model to rewrite verified training answers for the small assistant. Define extractable answer fields for torque values, units, and procedure prerequisites. Use a separate verifier for any reasoning or ordering that the fields cannot capture. Keep older equipment families and new manual sections separate for testing.
Compare ordinary supervised fine-tuning, same-family rewrites, and cross-family rewrites at matched training tokens and review effort. Evaluate new manual knowledge and retained procedures after each of three sequential updates. Examine failures on questions where old and new instructions conflict, since that is where both correct acquisition and retention matter.
Compare accepted-example rates and audit rejected rewrites by procedure type. Keep an equal-retention subset to check whether any apparent gain comes from dropping hard supervision rather than making it easier for the student to learn.
Sources
Original abstract
Continual post-training faces a dual challenge of acquisition and retention: models must learn new capabilities while preserving prior knowledge. While Self-Distillation Fine-Tuning (SDFT) mitigates this using self-supervision with in-context demonstrations, it relies on strong in-context learning and instruction-following abilities. Consequently, SDFT fails on smaller models critical for low-latency, on-device use. To enable continual learning at smaller scales, we first identify the root cause of failure. We show that poor acquisition and forgetting correlate linearly with the post-trained model's distribution shift, revealing the necessity of in-distribution learning. We thus introduce Synthetic Self-Distillation (SynthSelf), imitating SDFT's in-distribution rewrite by outsourcing the task to a minimally larger, same-family aligner. Coupled with off-policy supervision and rejection sampling, this provides the high-quality supervision small models require without breaking their original distribution. Evaluations spanning 4 acquisition datasets (including a mixture) and 14 retention benchmarks show SynthSelf consistently outperforms 6 methods (including off- and on-policy distillation) across 3 small model scales. Extensive further analyses confirm SynthSelf paves the way toward effective small-model continual learning.