Overview
Ask two language models the same question and you may get answers that are equally fluent and useful in different ways. Picking the better one means deciding what counts. A fact-checker may care most about accuracy. A teacher may care whether the reasoning helps a student. Prometheus 2 is an open-weight model designed to grade answers against criteria supplied by the person doing the evaluation. It can score an answer on its own or choose between two.
Key Findings
One reviewer can score every answer against a rubric. Another can look at two answers and choose the better one. Those sound like neighboring jobs until a developer tries to make one model do both and discovers that combining the training examples can weaken a skill it already had.
Two ways to grade
An absolute score answers a question such as “How well does this answer explain the evidence?” The grader receives a rubric with descriptions of what each score means. A pairwise judgment asks which of two answers better meets a specified criterion. An answer may be good enough on its own and still lose a head-to-head comparison. That distinction matters when one team wants to monitor the quality of a service over time and another wants to choose between two versions of it.
The authors fine-tuned separate copies of the same base model for the two grading formats. They then blended the trained parameters into a single model. The paper’s Equation 3 illustrates linear interpolation with scoring weights and comparison weights .
Here weights the scoring model. Appendix G describes the reported models’ DARE-Linear variant, which sparsifies and rescales changes from the base weights before combining them. The paper compares training on either format alone, training on both at once, and merging separately trained models. In Table 5, merging has the highest reported average direct-assessment correlation and average pairwise accuracy within each base-model family. Some individual columns favor other training methods.
What changed in the results
Pearson correlation measures how closely two sets of scores move together. Pairwise accuracy measures how often the evaluator picks the same answer as the reference judgment under the benchmark’s rules. These numbers answer different questions.
| Evaluator | FLASK correlation with human scores | Auto-J Eval pairwise accuracy, ties excluded |
|---|---|---|
| Mixtral-Instruct 8×7B | 0.420 | 73.50% |
| Prometheus 2 8×7B | 0.555 | 79.98% |
| GPT-4-1106 (Preview in Table 4) | 0.679 | 83.12% |
| Claude 3 Opus | 0.573 | 82.92% |
Selected results from Tables 3 and 4 of the EMNLP paper. The 0.555 on FLASK is a measure of agreement with human scores. It leaves room for substantial disagreement on individual answers. Auto-J Eval is in-domain for the Auto-J baseline, and Preference Bench, a separate benchmark in the paper, is in-domain for Prometheus 2. The fairest comparison depends on the task, reference evaluator, and training overlap.
The preference examples supply a reason
The Preference Collection supplies a criterion for each comparison and a written explanation alongside the preferred answer. Its 100,000 pairs span 1,000 criteria. The team pairs responses already graded in the Feedback Collection, assigns the winner using those grades, then asks GPT-4-1106 to explain how the pair differs. A reader can see the distinction in the paper’s Docker example, where two plausible answers are assessed against a specified standard. Section 3.3 This is a separate contribution from the weight merge. The criterion says what the evaluator should look for, while the explanation gives it a way to connect that standard to differences between responses. Without the criterion, a ranking example can teach the system a preference while leaving the basis for that preference ambiguous.
Two specialists can contribute to one model
Section 6 compares training on mixed formats with merging parameters from separately trained specialists. The separate treatment matters when direct grading and pairwise ranking put different demands on the same weights. The authors also vary the mix to study transfer across the two formats. The resulting model remains an evaluator whose judgments can be compared with people. The correlations and pairwise accuracy in the table tell different parts of that story.
Questions & Answers
The opening answers are simulated author responses based on the paper.
Q: What would I actually give the grader?
A: For direct assessment, we give it the request, the answer, and a rubric that describes what each score means. A reference answer can help specify what good looks like. In the paper’s pairwise evaluation, the model received two candidate answers and a criterion description, with no reference answer.
Q: Does higher correlation make a score interchangeable with a human score?
A: Correlation measures association. Two graders can move together while one systematically assigns higher scores, and a good aggregate correlation can conceal frequent disagreement near a pass threshold. A deployment test should measure both agreement and the cost of the errors that matter to the workflow.
Q: What does the merging experiment establish?
A: On the reported benchmarks, combining models trained for different grading formats helped more than merging copies trained for the same format in the Table 6 comparison. This supports the idea that the two training signals can complement each other. The mechanism and transfer to unrelated tasks remain open questions.
Q: How would you separate agreement with people from agreement with the training judge?
A: Use human-labeled examples from a new domain, hold the rubric fixed across both models, and report uncertainty on paired comparisons with the base model. Include the disagreements around the operating threshold. The published benchmarks motivate this test, while the threshold and deployment population are specific to the application.
Q: How much weight should we put on a small difference in the judge scores?
A: Check the metric and who provided the reference ratings first. The article cites Table 3's human-reference correlation for the 8×7B model. The 7B FLASK comparison is reported differently in Tables 3 and 5. Small gaps need uncertainty estimates and the two scoring tasks use different metrics. For deployment, collect human labels from the intended domain and rubric before ranking judges on a narrow margin.
Future Ideas
An appeal queue for rubric disagreements
For a low-stakes editorial or support workflow, compare a candidate answer with the current system’s answer to the same request. Score both separately, then run the pairwise judgment in both candidate orders. When the orders disagree, the direct scores tie, or the winning answers differ across formats, send the case, rubric, and answer evidence to a human. The same model can make correlated mistakes across both formats.
Start with a labeled holdout of real answers and a fixed review budget. Compare the queue with random sampling, near-threshold escalation, and repeated direct assessment at the same inference budget. Include an audit sample of cases the queue would accept automatically. Use paired uncertainty intervals for the share of material errors caught. Advance only if the queue catches more material errors at the same human workload across the kinds of answers the service handles.
Sources
Original abstract
Proprietary LMs such as GPT-4 are often employed to assess the quality of responses from various LMs. However, concerns including transparency, controllability, and affordability strongly motivate the development of open-source LMs specialized in evaluations. On the other hand, existing open evaluator LMs exhibit critical shortcomings: 1) they issue scores that significantly diverge from those assigned by humans, and 2) they lack the flexibility to perform both direct assessment and pairwise ranking, the two most prevalent forms of assessment. Additionally, they do not possess the ability to evaluate based on custom evaluation criteria, focusing instead on general attributes like helpfulness and harmlessness. To address these issues, we introduce Prometheus 2, a more powerful evaluator LM than its predecessor that closely mirrors human and GPT-4 judgements. Moreover, it is capable of processing both direct assessment and pair-wise ranking formats grouped with a user-defined evaluation criteria. On four direct assessment benchmarks and four pairwise ranking benchmarks, Prometheus 2 scores the highest correlation and agreement with humans and proprietary LM judges among all tested open evaluator LMs. Our models, code, and data are all publicly available.