How to measure AI performance: a beginner's guide in 6 steps
A step-by-step guide to measuring AI performance at work: set success criteria, build a test set, pick metrics, score outputs and compare with a baseline.

To measure AI performance, you decide what a good result looks like, build a small test set of real examples with known right answers, and score the AI's outputs against that set on accuracy, quality, speed and cost. Then you compare the score with a baseline (usually the way you did the job before) and re-run the same test whenever something changes. That's it. You don't need a data science team to start, and I'd argue the first useful version fits in a spreadsheet.
This guide walks through six steps and one full worked example. It's written for people using AI at work (a support team, a marketing lead, a solo founder) more than for researchers training their own models.
Before you start: the kit you need
- One specific task the AI does for you, such as sorting support emails or summarising sales calls.
- Between 30 and 100 real examples of that task, with personal data removed.
- The "right answer" for each example, written or checked by someone who knows the job.
- A spreadsheet with one row per example and columns for the AI's output and its scores.
- A rough idea of how long the task takes a person today.
If you're fuzzy on the basics, our explainer on what an AI model is covers the terms used below.
Step 1: Write down what success looks like
Vague goals produce vague scores. "The chatbot should be helpful" can't be measured; "the bot routes at least 90% of emails to the right queue and never invents a refund policy" can.
Anthropic's guide to defining success criteria gives a good model: instead of "classify sentiments well", it sets a target of an F1 score of at least 0.85 on a held-out set of 10,000 posts, a 5% improvement over the current baseline. You won't need 10,000 examples, but you will need a target number and a fixed set of cases to test it on.
Pick three to five criteria from this list:
- Accuracy: is the answer right?
- Quality: is it clear, on-brand and complete?
- Safety: does it avoid leaking private data or making things up?
- Speed: how long does a response take?
- Cost: what does each finished task cost in fees and staff time?
A criterion you can copy:
On our 50-email test set, the assistant must pick the correct category for at least 45 emails, include no made-up policy details in any draft, and reply in under 10 seconds.
Step 2: Build a small test set from real work
Pull examples from real tickets, documents or requests, and include the awkward ones: the angry customer, the email in two languages, the request that's half complaint and half question.
Anthropic's advice is blunt: "Prioritize volume over quality." In practice that means 50 decently labelled examples beat 10 perfect ones, because a single odd case can swing a tiny test by 10 points.
Can AI help write test cases? Yes, and OpenAI's evaluation best practices suggest mixing synthetic data with production data and expert-made examples. I'd still want a human to check every label, since a test written by the same model you're grading feels a bit circular to me.
A prompt for drafting extra edge cases:
Here are five real customer emails from our support inbox [paste]. Write 15 new emails in the same style that would be hard to classify: mixed topics, sarcasm, missing details, non-English phrases. Label each one as billing, bug, how-to or refund, and explain the label in one line.
Step 3: Choose metrics that match the task
The right metric depends on the shape of the output. Here's a quick map:
| Task type | Useful metric | Plain meaning |
|---|---|---|
| Sorting or labelling | Accuracy, precision, recall | How often it's right, and which mistakes it makes |
| Answering from your documents | Groundedness, citation accuracy | Does every claim trace back to a source? |
| Writing and summaries | Rubric score (1 to 5) | A person or second model grades against set rules |
| Anything live | Latency, cost per task, edit time | Speed, money and how much fixing it needs |
Precision and recall sound scary but aren't. Precision asks: when the AI said "refund", how often was it really a refund? Recall asks: of all the real refund emails, how many did it catch? The F1 score just blends the two into one number.
Public leaderboards like MMLU help with shortlisting, but I don't think they predict how a model will handle your invoices.
Step 4: Run the test and score the outputs
Run each example through the AI with the exact prompt and settings you use day to day. Paste the outputs into your spreadsheet. Then score them in one of three ways.
Exact match works for labels: the output either says "billing" or it doesn't, and a formula can check it. Human grading works for anything subjective, ideally with a written rubric so two people grade the same way. The third option, a second model acting as judge, is faster but needs checking against human scores first.
A judging prompt you can adapt:
You are grading a support reply. Score it from 1 to 5 on each of: correct answer, tone, and policy accuracy. Give 1 on policy accuracy if the reply states any policy that isn't in the reference text below. Reference policy: [paste]. Reply to grade: [paste]. Output only three numbers separated by commas, then one sentence explaining the lowest score.
Anthropic's own example code carries a note that it's generally best to grade with a different model from the one that wrote the output. That seems sensible. OpenAI adds that models are better at comparing two answers than rating one in a vacuum, so a side-by-side "which reply is better, A or B?" prompt is often more reliable than a 1 to 5 score.
Step 5: Compare against a baseline
A score on its own means very little. Is 82% good? It depends on what you had before.
Your baseline can be a person doing the task, the previous prompt, or a different model. Run the same test set through it and put the numbers side by side. For business tasks, I'd always include staff time, because an AI that's 95% accurate but needs five minutes of checking per item might not beat a human who takes four.
Step 6: Keep measuring after launch
Models get updated, prompts get tweaked and your customers change what they ask. OpenAI calls the habit of rerunning tests on every change "continuous evaluation", and it's the step most small teams skip. Re-run your frozen test set whenever you change the prompt, the model or the source documents, and add new failure cases from real use every month or so.
One practical warning: OpenAI is shutting down its hosted Evals platform. Existing evals go read-only on October 31, 2026, and the service is scheduled to close on November 30, so if your tests live there, export them now. A spreadsheet or one of the AI evaluation tools for production can take over.
Bigger organisations can add NIST's voluntary AI Risk Management Framework for the governance side.
A complete worked example: grading a support-email assistant
The figures below are illustrative, made up to show the method rather than taken from a real company.
A five-person online shop uses an AI assistant to sort incoming emails into billing, bug, how-to or refund, and to draft a reply for each. Before rolling it out, the support lead pulls 40 past emails, strips names and order numbers, and writes the correct category and an ideal reply for each one. That took her about two hours, which is honestly the least glamorous and most useful part.
She sets these targets: at least 90% correct categories, refund recall of at least 90% (missing a refund request is the costly mistake), no invented policy details, and an average reply score of 4 or higher.
The results come back mixed:
| Metric | Target | Result | Pass? |
|---|---|---|---|
| Correct category | 36 of 40 (90%) | 34 of 40 (85%) | No |
| Refund recall | 90% | 6 of 8 (75%) | No |
| Refund precision | None set | 6 of 7 (86%) | n/a |
| Drafts with invented policy | 0 | 5 of 40 | No |
| Average reply score | 4.0 | 3.9 | Close |
| Staff time per email | Under 6 min (old baseline) | 2.5 min | Yes |
Reading the misses is where the value is. Both missed refunds were polite emails that never used the word "refund" ("could I send this back?"). The five invented policies all described a 30-day return window, when the shop's real window is 14 days.
So she makes two changes: she adds the actual returns policy to the prompt with the line "only state policies found in the text below", and she adds three "send it back" examples to the instructions. On the re-run, the same 40 emails produce 37 correct categories, 8 of 8 refunds caught, and zero invented policies. Staff time stays near 2.5 minutes. That's a pass. And the time saving is the number she takes to her manager, which I suspect matters more to him than any F1 score.
Common mistakes and the fixes
Judging by feel
Trying five prompts and deciding "it seems fine" is what OpenAI's guide calls "vibe-based evals". The fix is boring: a fixed test set and a written score.
Testing only easy cases
Tidy examples flatter the AI. Add the messy ones on purpose.
Trusting an AI judge blindly
Grade 20 outputs yourself, compare them with the judge's scores, and keep adjusting the judging prompt until the two of you mostly agree on which replies are good and which aren't.
Changing the test set between runs
Grow it if you like, but keep a frozen core. Otherwise this month's score can't be compared with last month's.
Tips to get better results
- Score the cost of mistakes, not just the count: a wrong refund label probably hurts more than a clumsy greeting.
- Run each test twice; AI output varies between runs, and a 2-point swing may just be noise.
- Write the rubric before you see any outputs, so you aren't grading to fit.
When the numbers show a weak spot, our guides on prompt engineering and optimising AI models cover the usual fixes.
Next steps
Pick one task this week and build a 30-example test set for it. Once it exists, each later change is cheap to judge. Comparing models? Run the same set through two or three of them; our list of free AI models is a decent place to find candidates.
FAQ: how to measure AI performance?
How do you measure AI performance?
Set specific success criteria, collect real examples with known correct answers, run the AI on them and score the outputs for accuracy, quality, speed and cost. Compare with a baseline, then repeat after every change.
What are the most common AI performance metrics?
Accuracy, precision, recall and F1 score are the standard set for labelling tasks, while writing gets rubric scores from a person or a judge model, and live systems add latency, cost per task and editing time. You'll probably need three to five of them.
How many test examples do I need?
Thirty to 100 is a reasonable start for a work task. Below that, one odd case can move your score by several points. Anthropic's sample criterion uses 10,000 posts, but that's a production-scale target; a small team can learn plenty from 50, and adding more over time is easy once the spreadsheet exists.
Are public AI benchmarks useful for my business?
For shortlisting, yes. They won't tell you how a model handles your documents or customers, so I'd treat them as a first filter only.
Can I use AI to grade AI?
Yes, and it's much faster than grading by hand. Check it first, though: score around 20 outputs yourself, compare them with the judge model's scores and adjust its prompt. Using a different model from the one that produced the answers, and asking for A-versus-B comparisons, tends to give more reliable grades.



