Benchmark
A standard test set used to compare models — useful for direction, unreliable as a guarantee.
Benchmarks make models comparable on paper. Their weaknesses are structural: they leak into training data over time, they are optimised for as targets, and they rarely resemble your workload. Read them as a coarse signal and then run your own evals.
In practice: A model topping a coding benchmark and still failing on your codebase.