Learning & Reasoning / NeurIPS 2026

When the Answer Loses the Thread

A NeurIPS study finds that reasoning models can abandon a correct line of thought at the last step. Contrastive Thinking Decoding gives the trace more influence while leaving room to correct it.

Contrastive Thinking Decoding: Steering Answer Generation in Reasoning Models

Taehyeon Kim, Hyunsoo Lee, Yu Jin Kim, Youngsoo Jang, Moontae Lee

Overview

A reasoning model can work through a problem correctly and still give the wrong answer. The reverse can happen too. A flawed trace can be followed by a correct final answer. This paper measures that handoff and tests a decoding rule that uses the trace as evidence for the answer without turning it into a script.

Key Findings

Hold the trace fixed and measure what changes

Figure 1 captures the handoff the paper studies. Qwen3-8B reaches 19 in its thinking trace, then recomputes and outputs 107 in the answer phase. A single example does not establish a failure rate. The experiment makes that step observable by generating a trace once, keeping it fixed, and resampling only the answer phase. The last boxed answer before the end of thinking supplies the trace prediction. The final boxed answer supplies the response prediction.

Qwen3-8B concludes 19 in its trace, then produces 107 in the answer phase after recomputing the problem.
Qwen3-8B concludes 19 in its trace, then produces 107 in the answer phase after recomputing the problem.

For trace-answer pairs with an extractable trace conclusion, disagreement can be written as

D=1∣I∣∑i∈I1[a^ianswer≠a^itrace].D=\frac{1}{|\mathcal I|} \sum_{i\in\mathcal I}\mathbf 1[\widehat a_i^{\mathrm{answer}}\ne\widehat a_i^{\mathrm{trace}}].

A trace with no boxed answer is excluded. A missing final box counts as disagreement if the trace has one. For the clean-trace audit the authors additionally require natural termination. At 32K, disagreement remains 6.4–48.2% across the audited settings. They exclude Qwen3-1.7B from that specific clean-trace comparison because its traces frequently omit the boxed answer. The metric compares conclusions, so two traces with different derivations but the same conclusion can agree by this measure.

The budget sweep in Table 1 shows why better trace reasoning does not translate directly into better final answers. For R1-Distill Qwen 7B on AIME 2024, expanding the budget from 4K to 32K raises the rate of solving the problem in the trace by 26.45 points, while final Pass@1 rises by 1.88 points. The columns have different denominators and answer different questions.

Maximum thinking budgetSolved in traceTrace–answer disagreementFinal Pass@1
4K19.69%83.33%49.58%
8K34.17%56.67%49.48%
16K45.73%36.67%50.00%
32K46.14%28.17%51.46%

Separate useful corrections from harmful drift

A trace-copy baseline returns the trace's extracted answer when one is available. Standard decoding beats copying in both configurations shown in Table 2, and CTD improves on standard decoding in those rows. The difference between copying and ordinary decoding tells us that the answer phase does useful work on some problems. It can also override correct traces.

AIME 2024, 32KTrace-copy baselineStandard decodingCTD
Qwen3-8B68.8%75.9%77.7%
R1-Distill 7B48.5%51.5%57.7%

The four-way analysis makes the distinction explicit. A recovery starts from an incorrect trace and ends with a correct answer. Drift starts from a correct trace and ends with an incorrect answer. Across paired draws, the change in accuracy between the trace and the answer is

Accanswer−Acctrace=nrecovery−ndriftN.\mathrm{Acc}_{\mathrm{answer}}-\mathrm{Acc}_{\mathrm{trace}} =\frac{n_{\mathrm{recovery}}-n_{\mathrm{drift}}}{N}.

For R1-Distill 7B under CTD-Null on AIME 2024, Table 6 reports 46 recoveries and two drifts over 480 runs, a net 9.2-point rise from trace accuracy to final-answer accuracy in that condition. That is a within-CTD decomposition. The base decoder can also have low drift, so these counts do not give a causal estimate of the improvement from switching decoders.

Contrast evidence from two traces in the same model

CTD scores each answer token under two contexts. One contains the primary reasoning trace. The other contains a deliberately weakened reference, such as a null prompt or a prompt directed toward an opposite approach. Both passes use the same model and generated answer prefix. If vv is the next token, the rule in Section 3 is

ℓ~t(v)=ℓtP(v)−λℓtN(v),qt=softmax⁡ ⁣(ℓ~t/τ).\widetilde{\ell}_t(v)=\ell_t^{P}(v)-\lambda\ell_t^{N}(v), \qquad q_t=\operatorname{softmax}\!\left(\widetilde{\ell}_t/\tau\right).

Here PP and NN denote primary and noisy contexts, λ\lambda controls how much reference score to subtract, and τ\tau is the decoding temperature. The authors reuse λ=0.15\lambda=0.15 across their evaluation. A token gains relative weight when the primary trace supports it more strongly than the reference does. A continuation that both contexts make likely receives less of an advantage. The adjustment acts on the distribution at each step, leaving the answer phase free to generate a different conclusion.

The primary and noisy traces feed two answer-phase scoring passes in the same model. Their contrast steers the next token.
The primary and noisy traces feed two answer-phase scoring passes in the same model. Their contrast steers the next token.

Figure 3 pairs total generated tokens with disagreement across budgets. In the paper's AIME diagnostic, CTD reduces disagreement at the tested operating points. That makes the transition more consistent in these runs. Whether consistency also improves accuracy still depends on the quality of the primary trace and on which disagreements have been removed.

On AIME 2024 and 2025, the disagreement curves can be read alongside the generated-token bars for the base, null-reference, and opposite-reference settings.
On AIME 2024 and 2025, the disagreement curves can be read alongside the generated-token bars for the base, null-reference, and opposite-reference settings.

Charge for the second pass before comparing efficiency

Counting only emitted answer tokens would understate the cost of CTD. Figure 4 charges primary thinking, noisy-reference prefill, and two answer-phase evaluations per emitted token. In the figure's accounting, the approximate total is

CCTD≈Nthink+Nreference prefill+2Nanswer.C_{\mathrm{CTD}} \approx N_{\mathrm{think}}+N_{\mathrm{reference\ prefill}}+2N_{\mathrm{answer}}.

This is the paper's token-equivalent accounting, not a claim that every token has identical latency. The two answer evaluations can run in one batch. At large thinking budgets the answer occupies a small share of computation, and shorter emitted answers can offset the added pass. The paper measures latency on a single H200 with vLLM. Its result is specific to that setup.

The six panels compare Pass@1 with fully charged token-equivalent compute. The AIME curves show the clearest separation, while the MATH500 curves are closer.
The six panels compare Pass@1 with fully charged token-equivalent compute. The AIME curves show the clearest separation, while the MATH500 curves are closer.

Read the cross-task results and uncertainty together

The strongest paired evidence comes from GPQA Diamond. Table 4 reports a statistically significant gain at both Qwen3 scales, based on 198 questions and paired McNemar tests with p<0.001p<0.001. Social-pressure resistance and general knowledge show smaller changes. In code, the opposite reference works more consistently than the null reference, so the choice of reference is consequential.

BenchmarkModelBaseCTDChange
GPQA DiamondQwen3-8B57.6%63.6%+6.0 points
GPQA DiamondQwen3-32B68.2%72.7%+4.5 points
SycophancyQwen3-8B30.4%32.0%+1.6 points
SycophancyQwen3-32B36.4%38.8%+2.4 points
MMLUQwen3-8B82.8%83.1%+0.3 points

The AIME headline scores and the matched-pipeline bootstrap in Appendix H come from different comparisons. In the latter, the Qwen3-8B AIME 2025 estimate at 16K is −0.6 points, with a 95% interval from −4.4 to +3.1. The other matched-pipeline AIME intervals also include zero. The result supports inspecting the answer phase and testing CTD where disagreement is high. It does not support expecting a fixed gain on every reasoning benchmark. Agreement with a written trace also does not establish that the trace faithfully describes the model's internal reasoning.

Questions & Answers

Q: Why can a correct trace still end in a wrong answer?

A: The model generates the final answer in a separate phase. It can recompute part of the solution, attend to a different cue in the question, or choose a conclusion that differs from the trace. The same flexibility sometimes repairs a trace error.

Q: Does the method need a critic or new training?

A: The method uses one model with fixed weights. It changes how next-token scores from two trace-conditioned contexts are combined during answer generation.

Q: Does less disagreement always mean better answers?

A: Agreement can preserve a mistaken trace. Final-answer accuracy and the separate counts of recoveries and drift are needed to distinguish useful consistency from repeating an error.

Future Ideas

Learn when to use the second pass

Use a small, independently labeled calibration set to test whether a model's uncertainty at the transition predicts when contrastive decoding will help. Gate the second pass on that signal and compare with always-on CTD, ordinary decoding, and trace copying at matched total inference cost.

Measure accuracy, recovery, drift, and tail latency on held-out math, code, and science tasks. Include adversarial traces and cases whose final conclusion is absent or ambiguous. A useful gate should preserve the improvements on high-disagreement cases while spending less on cases ordinary decoding already handles.

Sources

Original abstract

Large reasoning models separate inference into an explicit thinking phase followed by a final answer phase, yet how the answer phase uses the reasoning trace remains unclear. We present a systematic study of this trace-to-answer transition across multiple LRMs and find that (i) final answers diverge from correct reasoning traces at rates between 6% and 48%, even when traces terminate cleanly at full budget, (ii) naively copying the trace answer underperforms standard decoding in some settings, revealing that the answer phase has independent corrective capacity, and (iii) the same phenomenon extends beyond structured tasks to settings where reasoning traces and final answers can diverge under social-pressure framing (sycophancy). These findings motivate Contrastive Thinking Decoding (CTD), a training-free, single-model decoding method that contrasts answer-phase logits under the primary reasoning trace against those from a deliberately degraded noisy trace. CTD selectively amplifies trace-consistent tokens while preserving the answer phase’s corrective capacity, concentrating CTD-internal joint mass on the both-correct cell (T✓A✓) without inducing trace-correct-to-answer-wrong drift relative to the base distribution. Across math reasoning, science, code, social-pressure resistance (sycophancy), and a broad knowledge benchmark (MMLU), CTD reduces trace-answer disagreement and improves the accuracy–compute trade-off without parameter updates or auxiliary models at decoding time.