Overview
A model may be good at following instructions and unreliable at finding evidence. Giving it one score for both can hide the very problem you want to fix. The BiGGen Bench tests nine capabilities with 765 questions and tasks, each paired with its own scoring criteria. The authors used those criteria to evaluate 103 language models, then checked how closely five machine evaluators agreed with people.
Key Findings
Imagine asking two assistants to use a tool. One gives a polished description of the tool and the other actually completes the requested operation. An evaluator needs to recognize the difference. The benchmark starts with such distinctions and builds a scoring guide for each individual case.
What the rubric can see
A generic rubric might reward “helpfulness.” An instance-specific rubric spells out which parts of a task an answer has completed. In a math example from Figure 8, the rubric checks for a particular substitution, rather than simply asking whether the reasoning seems sound. The paper’s Figure 1 illustrates what changes when a grader receives that finer description.
Section 3.2 describes a first set of 385 human-authored instances and the use of GPT-4-0125 to generate more candidates. Coauthors checked prompts, reference answers, and rubrics. The main paper reports that five instances were removed after cross-validation, leaving 765 across 77 tasks. The nine capabilities include reasoning, safety, grounding, tool use, and multilingualism. Here a capability is the broad thing being measured, a task is a class of questions, and an instance is one scored example.
How close are the graders to people?
Table 3 reports human judgments on 3,236 model responses across 765 inputs. Its caption and the counts in Appendix B.2 differ, so the number here follows the table that reports the correlations. Pearson correlation compares how closely the human ratings and evaluator scores rise and fall together.
| Grading method | Average Pearson correlation with human ratings |
|---|---|
| Prometheus 2 8×7B | 0.471 |
| Prometheus 2-BGB 8×7B | 0.577 |
| Prometheus 2-BGB, three samples with self-consistency | 0.607 |
| GPT-4-Turbo-2024-04-09 | 0.623 |
| Majority voting across evaluator LMs | 0.627 |
Selected rows from Table 3 of the NAACL paper. The five-sample self-consistency row also reports 0.607. Prometheus 2-BGB was trained using GPT-4-1106 feedback on benchmark inputs. The 0.004 gap between majority voting and the highest individual evaluator is a reported point difference. The practical question is whether an evaluator is dependable on the particular capability, language, and kind of failure the team cares about.
Section 5.2 varies the rubrics themselves. With the benchmark’s instance-specific criteria, the tested evaluator judgments track human ratings more closely than with generic MT-Bench or task-level FLASK guidance. This checks the contribution of the rubric separately from replacing the evaluator model. Figure 6
What the benchmark leaves open
An offline test gives every model a comparable set of tasks. It captures a finite slice of what happens in live use. Answers can also change with sampling, and a model-based grader can bring its own preferences. The benchmark is most informative when its capability-level results are read alongside individual failures and human scores.
How much does a long answer help its own score?
The verbosity study uses 78,795 judgments from GPT-4-Turbo across 103 response models. The researchers look at token length against assigned score and run an ablation to test the effect more directly. The paper reports a correlation of 0.05 between token count and assigned score. With nearly 79,000 judgments, a small association can still be statistically detectable. The aggregate result leaves room for a stronger effect on a particular capability or prompt. Figure 7
Reading across the capability map
The study also compares base, post-trained and proprietary models to see how the profiles change with training and scale. The same headline score can hide a gain in one skill and a shortfall in another. That is where the benchmark’s task and capability breakdown is most useful. Teams can return to the individual rubrics to see whether their interpretation of a low score matches the evidence the grader used.
Questions & Answers
The opening answers are simulated author responses based on the paper.
Q: Why does a question need its own rubric?
A: We want evaluators to recognize which details matter for that example. A broad instruction to judge reasoning leaves the evaluator to decide what a complete solution looks like. A detailed rubric makes that decision easier to inspect and compare across graders.
Q: Does training a judge on model feedback tell us how people will score?
A: The adapted judge was fine-tuned on 50,490 GPT-4-1106 judgments according to Appendix E. The comparison with human ratings is a separate measurement on benchmark prompts. Table 3 shows substantial variation between capabilities. Human validation on a new domain would still be needed.
Q: Would more examples settle which grader is best?
A: More examples can reduce sampling uncertainty when they cover the intended distribution. Correlation can still depend on which models answer the prompts and the range of scores those models earn. A useful comparison would stratify by capability and language, then test whether the ranking survives changes to those choices.
Q: Who sets the reference when a benchmark measures agreement with people?
A: The paper reports human-reference correlations, but crowd raters qualified partly through agreement with GPT-4-1106. The adapted judge also used model-generated feedback. That context matters when interpreting small differences and claims of independent agreement. The number of ratings reported in Table 3 and Appendix B.2 differs, and the adapted judge's response-model split warrants clarification before making a strong claim about unseen models.
Future Ideas
Turn one real failure into a small family of tests
Give a product team a way to turn a verified failure into two controlled variants and a rubric. For a grounding failure, one variant changes a relevant fact in the evidence while another changes an irrelevant detail. The answer should follow the first change and ignore the second. A reviewer sees the three answers together and checks whether the automated score reflects both behaviors.
Pilot with a small consented set of human-verified failures from one product. Define the expected response to each variant before scoring, and keep all variants of a failure together when splitting development and holdout sets. First measure whether the grader follows relevant changes and ignores irrelevant ones. Then compare regression detection between two checkpoints against an equally sized set of independently written questions. Report uncertainty by failure family, and retire variants whose expected answer remains ambiguous in human review.
For a replication, qualify raters against a human-authored rubric key and hold out both prompt families and response models. Keep newly written variants and the failure they came from in the same split.
Sources
Original abstract
As language models (LMs) become capable of handling a wide range of tasks, their evaluation is becoming as challenging as their development. Most generation benchmarks currently assess LMs using abstract evaluation criteria-like helpfulness and harmlessness-which often lack the flexibility and granularity of human assessment. Additionally, these benchmarks tend to focus disproportionately on specific capabilities such as instruction following, leading to coverage bias. To overcome these limitations, we introduce the BiGGen Bench, a principled generation benchmark designed to thoroughly evaluate nine distinct capabilities of LMs across 77 diverse tasks. A key feature of the BiGGen Bench is its use of instance-specific evaluation criteria, closely mirroring the nuanced discernment of human evaluation. We apply this benchmark to assess 100 frontier LMs using five evaluator LMs. Our code, data, and evaluation results are all publicly available at https://github.com/prometheus-eval/prometheus-eval.