Building better AI benchmarks: How many raters are enough?

Google Research explores the trade-off between number of items and human raters per item to improve AI benchmark reproducibility and capture the nuance of human disagreement.

Raccontata daresearch.google