Using a strong language model to grade outputs, either scoring one answer against a rubric or picking the better of two. It scales the evaluation of open-ended text, where exact-match metrics fail, but the judge has biases you have to measure against human labels.
Card 1 of 292. Answer: Using a strong language model to grade outputs, either scoring one answer against a rubric or picking the better of two. It scales the evaluation of open-ended text, where exact-match metrics fail, but the judge has biases you have to measure against human labels.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
The common formats are single-answer grading, pairwise comparison and grading against a reference answer. Zheng and colleagues, who built MT-Bench and Chatbot Arena, reported in 2023 that GPT-4's verdicts agreed with human experts over 80% of the time, about as often as the experts agreed with each other. That is why judges are now standard for comparing prompts, models and RAG pipelines during development.
Example: before trusting a judge, compare it with human labels on a sample. On 200 answer pairs, humans prefer A in 140 and the judge picks A in 150, and they agree on 170, which is 85%. But a judge that always said A would already agree 70% of the time, so compute Cohen's kappa: chance agreement is 0.7 × 0.75 + 0.3 × 0.25 = 0.60, and κ = (0.85 − 0.60) / (1 − 0.60) ≈ 0.63.
Known biases: position bias, preferring the first answer shown; verbosity bias, preferring longer answers; and self-enhancement, preferring outputs from its own model family. Mitigations: run each pair in both orders, give a specific rubric, ask for the reasoning before the score, and judge with a different model family from the ones being compared.
