What are AI evals?
Also called: evals, LLM evaluation, AI evaluation
AI evals are tests that measure how well an AI system performs on a set of example inputs with known good answers. They tell you whether a change to a prompt, model or retrieval setup actually made things better, or quietly made them worse.
How do AI evals work?
You collect realistic test cases, run the system on all of them, and score the outputs automatically, by people, or with another AI as a grader, then compare scores across versions.
Why do AI evals matter?
AI output varies, so 'it worked when I tried it' isn't enough. Evals are how serious teams ship AI features with confidence.
An example
Example: Before switching models, a support team runs 200 past customer questions through both and compares accuracy and tone scores.
Related terms
Learn it properly
AI evals are covered in AI Builder, with short lessons, practice questions and instant feedback.