Benchmark · general

RewardBench

0 results 0 models

RewardBench is the first benchmark and leaderboard for evaluating reward models used in RLHF; it measures how often a reward model scores the human-preferred ("chosen") response above the dispreferred ("rejected") one, reported as accuracy.

Example: A prompt paired with two candidate completions — one preferred and one dispreferred — drawn from categories such as general Chat, adversarial Chat Hard, Safety (refusal vs. compliance), and Reasoning (math/code); the model must rank the preferred completion higher.
Scoring: The metric is pairwise accuracy — the share of examples where reward(chosen) > reward(rejected). Accuracy is computed per prompt, weighted-averaged within each of the four categories, then averaged across categories into one overall score.
Verification: A single item counts as correct only when the reward model assigns a strictly higher scalar reward to the chosen response than to the rejected one; results are aggregated over the fixed ~2,985-item dataset and reported on the public leaderboard for reproducible comparison.
Why it matters: Reward models steer RLHF alignment but were long evaluated only indirectly; RewardBench gives a direct, reproducible measure that surfaces failure modes on hard, safety-critical, and reasoning preferences.

Worked example

Task

One trio: Prompt — "Explain why the sky appears blue." Chosen — an accurate response describing Rayleigh scattering of the shorter (blue) wavelengths. Rejected — a plausible but incorrect response (e.g., "because it reflects the ocean"). The reward model must produce a scalar score for each (prompt, response) pair.

Solution

Scored correct if and only if r(prompt, chosen) > r(prompt, rejected). The item is a "win" when the Rayleigh-scattering response (chosen) receives the higher scalar reward; it adds 1 to the category's accuracy count (0 otherwise).

Walkthrough

The chosen response is the human-preferred, factually correct one, so a well-aligned reward model should rank it above the incorrect rejected response; grading is a pure pairwise comparison of the two scalar rewards (higher on chosen = correct), and such wins are weighted-averaged within the category and then across categories.

No verified scores reported yet for this benchmark.