08 / Trust & Governance
Benchmark
A public, standard test used to compare models. Useful for a shortlist, a poor guide to how a model will do on your own work.
How you’ll hear it
“It tops every benchmark.”
“Good. Now test it on our cases.”
Avoid
Choosing a model on benchmark scores alone.
The longer version
Benchmarks let providers and researchers compare models on common tasks. They say little about your documents, customers and edge cases, and models can be tuned to score well on them. Treat them as a filter, then run your own evals.