...evaluate and trust AI
Jump to navigation
Jump to search
| AI Solver map |
|---|
|
Your current page is in bold
0 · Start
1 · Predict & analyze
tables, numbers, events
Predict values
Predict categories Forecast time series Cluster / segment Find outliers Make recommendations Find key drivers 2 · Understand unstructured data
text, documents, images, audio
3 · Generate & converse
4 · Decide & act
5 · Build it right
applies to every path
|
YouTube ... Quora ...Google search ...Google News ...Bing News
- AI Solver ... Train, Validate, and Test ... Evaluating Machine Learning Models ... Administration ... NIST
Use this page on every path. Decide what "good" and "unacceptable" mean before you build, then measure both continuously. A repeatable evaluation set is the most valuable asset in an AI project: it lets you pick models, catch regressions and switch vendors with confidence.
Start here
- Classic machine learning (prediction, classification, clustering) → to verify your chosen algorithm is performing optimally, link evaluation to Train, Validate, and Test workflows:
- Holdout validation — split data into training and test sets to measure generalization
- Cross-validation — use K-fold splits to check that performance is stable across subsets
- Time-based splits for anything time-ordered (never shuffle the future into training)
- A baseline — every model must beat a simple rule, the mean, or last period's value
- LLM and generative applications → build an eval set: start with 30–100 real cases and grow it from production failures
- Code-based checks — exact match, valid JSON schema, correct tool arguments, unit tests pass
- LLM-as-a-judge — a model grades outputs against a written rubric; calibrate it against human ratings
- Human review — for subjective quality and high-stakes outputs
- Online — A/B tests, user feedback, sampled production traces
- RAG → retrieval recall@k, groundedness (every claim supported by retrieved text), citation accuracy, and correct "I don't know" answers
- Agents → task success rate, cost and time per task, tool-call correctness, human interventions, safety violations; read full trajectories
Metrics by task
| Task | Metrics |
|---|---|
| Classification | Accuracy, precision, recall, F1-score, ROC-AUC, PR-AUC, confusion matrix, calibration |
| Regression | MAE, RMSE (mean squared error family), MAPE/WAPE, R-squared, interval coverage |
| Forecasting | MASE, WAPE, quantile loss / CRPS, bias, rolling backtests |
| Clustering | Silhouette, Davies–Bouldin, stability, business usefulness |
| Anomaly detection | Precision of top alerts, recall on known incidents, false alarms per day |
| Recommendation | Precision@k, NDCG, coverage, diversity; online click-through and conversion |
| Vision | mAP (detection), IoU / Dice (segmentation), latency on the target device |
| Speech | Word error rate, latency, naturalness ratings |
| Generative and agentic | Instruction-following rate, tool-call correctness, groundedness, LLM-as-a-judge scores, task success, cost per task |
Trust: risks and controls
| Risk | Controls |
|---|---|
| Wrong or made-up answers (hallucination) | Ground in retrieved sources, cite them, allow "I don't know", test on known questions |
| Bias and unfair outcomes | Measure error rates per group (e.g., Fairlearn), representative data, human review of decisions about people |
| Privacy | Collect less, redact personal data, control retention, keep sensitive work on-premises or on-device |
| Security: prompt injection, data leaks, over-powered agents | Treat retrieved content as data, least-privilege tools, sandboxes, human approval for irreversible actions (OWASP Top 10 for LLM Applications) |
| Harmful or off-brand content | Input/output guardrails and classifiers (e.g., Llama Guard, NeMo Guardrails, Azure AI Content Safety, Amazon Bedrock Guardrails) |
| Silent degradation | Monitor quality and drift; re-run evals on every model or prompt change |
| Opaque decisions | Explanations (SHAP), model cards, decision logs, an appeal path for affected people |
Governance and regulation (check current status)
- NIST AI Risk Management Framework — voluntary US framework (govern, map, measure, manage), with a Generative AI profile
- ISO/IEC 42001 — certifiable AI management-system standard
- EU AI Act — bans on prohibited practices (since February 2025) and obligations for general-purpose AI models (since August 2025) already apply. The Digital Omnibus, in force since July 2026, moved high-risk obligations to 2 December 2027 for stand-alone systems (such as hiring, credit scoring, education) and 2 August 2028 for AI built into regulated products. Digital Omnibus: new AI Act deadlines | CASRAI ... AI Act | European Commission
Real-world examples
- Evals before rollout — Morgan Stanley ran evaluations with its advisors and experts before scaling its assistant, and credits them for adoption by more than 98% of advisor teams. Morgan Stanley | OpenAI
- Accountability — in Moffatt v. Air Canada (2024), a tribunal held the airline responsible for its chatbot's wrong answer, rejecting the argument that the chatbot was responsible for its own statements. Moffatt v. Air Canada | McCarthy Tétrault
- Perception vs. measurement — in METR's 2025 trial, experienced developers were 19% slower with AI tools while believing they were faster. Measure; don't assume. METR study | arXiv
Learn more
- OpenAI Evals
- promptfoo: test and red-team LLM apps
- Ragas: RAG evaluation
- Inspect: evaluation framework | UK AI Security Institute
- Langfuse: open-source LLM observability and evals
- Train, Validate, and Test ... Evaluating Machine Learning Models
Where next? ◀ Build it right · ...choose a model · ...deploy AI · ...build an agent · ▶ Back to all goals