How we explain AI benchmarks

This page explains, in plain English, how to read the benchmark results and leaderboards on Fanetti AI. Our leaderboard pages link here so you can always check what a number means before you rely on it. New to the topic? Start with What is an AI benchmark?

What common benchmarks measure

Each benchmark is a fixed test that looks at one kind of skill. The main families are:

Knowledge and reasoning. Large sets of multiple-choice or short-answer questions across many subjects. MMLU is the best-known example; harder sets written by experts are used now that many top models score near the top of MMLU.
Math. Word problems and competition-style questions where the final answer can be checked exactly.
Coding. The model writes or fixes code, and automated tests check whether it works. Results are usually given as pass@1: the share of problems solved on the first try.
Human preference. People compare two anonymous answers and pick the better one. The votes become an Elo-style rating. This captures how helpful and pleasant answers feel, not whether they’re correct.
Speed and price. How fast a model responds and what it costs per token. These depend on the provider and settings as much as on the model.

Why scores differ between sources

You may see the same model with different scores for the same benchmark on different sites. Common reasons:

Different settings. The exact prompt wording, the number of examples shown first, how answers are extracted, and how many tries are allowed all change the score.
Different model versions. Providers update models, sometimes under the same name. A score is only valid for the version and date it was measured.
Different subsets. Some sources run a full test, others a sample or an updated edition.
Who ran it. A self-reported score comes from the model’s maker, who chooses the settings. Independent measurements use one consistent setup for every model, which makes them easier to compare.

That’s why we always show where a number came from and when it was measured, and why you should compare numbers only within the same source.

How a “best” model is generally judged

There is no single “best” AI model. A model that leads one leaderboard may be average on another. When people call a model the best, they usually mean it does well across several independent benchmarks and is preferred by people in side-by-side comparisons, at a reasonable speed and price.

For your own use, “best” means best at your tasks, within your budget and privacy needs. Use rankings to build a shortlist, then test it yourself. Our guide How to pick an AI model walks through that.

Caveats to keep in mind

Contamination. If test questions leak into a model’s training data, it can score well by memory rather than skill. See contamination.
Saturation. When most top models score close to the maximum, a benchmark stops telling them apart.
Small differences. A gap of a point or two is often within normal run-to-run variation and may not be noticeable in real use.
Self-reported results. Treat a maker’s own numbers as claims until an independent source confirms them.
Benchmarks aren’t your work. No public test perfectly matches what you’ll use a model for.
Things change quickly. Always check the date next to a result.

Sources and credit

When our pages show benchmark results, each one names its source, links to it, and shows the date it was measured. We follow each source’s license and terms: some data may be shown with credit, and some sources we only name and link to rather than republish. Independent trackers you may come across include Artificial Analysis, Epoch AI, and Arena.

If you spot a number that looks wrong or out of date, please let us know so we can check it against the source.