Checklists

From PRIMO.ai
Jump to navigation Jump to search

YouTube search... ... Quora search ...Google search ...Google News ...Bing News


A checklist is a short list of the critical steps or checks for a task, used at a defined moment, called a pause point, so that nothing important gets skipped. In AI work, checklists cover the project lifecycle, data, model evaluation, production readiness, fairness and ethics, security, research reproducibility, and reporting in regulated fields such as health care. A checklist is not a textbook or a best-practice manual. It is a memory aid and a coordination tool for people who already know what to do but, under time pressure and across team hand-offs, sometimes don't do it.

Where checklists came from. The modern pilot's checklist dates from 30 October 1935, when Boeing's Model 299 bomber prototype crashed on takeoff at Wright Field, Ohio, because the crew forgot to release the gust locks that hold the control surfaces in place while parked. Test pilot Maj. Ployer "Pete" Hill died. The response was a simple preflight checklist, and the aircraft went on to become the B-17. Medicine followed decades later. The World Health Organization's Surgical Safety Checklist, piloted in eight hospitals in eight cities, cut deaths from 1.5% to 0.8% and major complications from 11.0% to 7.0% (New England Journal of Medicine, 2009). Atul Gawande's The Checklist Manifesto (2009) popularized the design rules behind such checklists, many drawn from Boeing flight-deck designer Daniel Boorman.

Checklists in AI and machine learning. The field adopted the idea for its own recurring failures: Google's ML Test Score rubric for production readiness (2017); Datasheets for Datasets (Gebru and colleagues, 2018) and Model Cards (Mitchell and colleagues, 2019) for documentation; the NeurIPS reproducibility checklist introduced by Joelle Pineau and colleagues (2019); Microsoft Research's co-designed AI fairness checklist (Madaio, Stark, Wortman Vaughan and Wallach, CHI 2020 best paper); CheckList for behavioral testing of NLP models (Ribeiro and colleagues, 2020); and reporting standards such as REFORMS and TRIPOD+AI (2024).

The problem it addresses. AI projects fail in predictable ways: information leaking from test data into training, data that does not represent real users, metrics that do not match the decision being made, bias that nobody measured, missing monitoring, and insecure integrations. The steps that prevent these failures are well known. They are also easy to skip when deadlines loom, when work passes between data engineers, scientists and operators, or when a demo looks good enough.

The central mechanism. A checklist makes the critical steps explicit, checks them at defined moments, names who confirms each one, and ideally links each answer to evidence. It turns tacit knowledge into a shared routine and makes omissions visible before they become failures.

How the practice has changed.

  • From personal tip lists to community standards. Early checklists were blog posts. Today conferences require paper checklists, journals require reporting checklists, and regulators publish assessment lists.
  • From ticked boxes to evidence and automation. Good AI checklists ask for an artifact (a test result, a datasheet, a sign-off), and items that code can check become automated gates.
  • From documentation to measurement. Checklists now double as evaluation instruments: physician-written criteria score AI answers to health questions, model-generated yes/no checklists help AI judges grade outputs, and checklist scores are even used as training rewards.
  • How they differ from frameworks. Frameworks such as the NIST AI Risk Management Framework describe what good looks like. Checklists make sure specific steps actually happened at specific times.


A good checklist does not make experts smarter; it makes sure no one skips the few steps that cause most failures. In AI work, short checklists tied to clear pause points, named owners and real evidence catch the errors that sink projects, while long box-ticking lists only create false confidence.


Checklists in Practice

How a checklist works in practice:

  1. Define the pause points. Decide where the team stops to check. For AI work: problem framing, data ready, before training, before release, after deployment, and after any incident. The WHO surgical checklist uses three: before anesthesia ("sign in"), before the first incision ("time out") and before the patient leaves the operating room ("sign out").
  2. Choose the type. A read-do checklist is followed item by item, like a recipe, and suits new or rarely performed tasks such as setting up an evaluation pipeline. A do-confirm checklist lets experts work from memory and then stop to confirm, and suits release reviews.
  3. Pick the killer items. Include the steps most likely to be skipped and most costly when skipped. Boorman's guidance, as reported by Gawande: usually five to nine items per pause point, one page, the team's own wording, and testing in real use.
  4. Tie each item to evidence. "Leakage checked" becomes "split method documented, duplicate check across splits run, link to result."
  5. Assign roles and authority. Name who reads the list, who confirms each item and who can stop a release. In Peter Pronovost's central-line checklist at Johns Hopkins, which Gawande describes, nurses were authorized to stop a doctor who skipped a step; a checklist that nobody can enforce is only a suggestion.
  6. Automate what code can check. Schema validation, overlap between training and test sets, feature unit tests and evaluation thresholds can run in a continuous integration pipeline. The ML Test Score gives a full point only for tests that run automatically and repeatedly, and half a point for documented manual checks.
  7. Review and revise. Update the checklist after incidents and near misses, and drop items that never catch anything.
Checklist family What it checks Examples Typical pause point
Project lifecycle Framing, data, modeling, launch and maintenance Géron's eight-step ML project checklist; fast.ai data project checklist Kickoff and stage reviews
Data documentation Motivation, composition, collection, consent, intended uses Datasheets for Datasets; deon's data sections Data ready
Model documentation Intended use, evaluation across groups, limitations Model Cards Before release
Production readiness Data, model, infrastructure and monitoring tests ML Test Score (28 tests) Release gate
Behavioral testing Capabilities and failure modes beyond held-out accuracy CheckList (minimum functionality, invariance and directional tests) During evaluation
Research reproducibility Code, data, splits, statistics, leakage NeurIPS paper checklist; ML Code Completeness Checklist; REFORMS Before publication
Clinical AI reporting Study design, data, validation, fairness, openness TRIPOD+AI (27 items) Before publication or purchase
Responsible AI Fairness, transparency, oversight, accountability Microsoft's co-designed fairness checklist; EU ALTAI; deon Kickoff and release
Security and governance Threats, asset inventory, red teaming, legal and regulatory duties OWASP LLM Applications Cybersecurity and Governance Checklist; OWASP Top 10 for LLM Applications Design and release
Evaluation rubrics Whether each requirement of an output is met HealthBench physician criteria; generated checklists for AI judges Every evaluation run

Major components. A pause point, a short list of items, an owner for each item, the evidence required, an escalation path when an item fails, and a version history so the list can improve.

Dependencies and conditions. Checklists need organizational support. In Madaio and colleagues' co-design study with 48 practitioners, AI fairness checklists could give "organizational infrastructure for formalizing ad-hoc processes" and empower individual advocates, but organizational culture shaped whether they worked. Teams also need time budgeted for the checks, the authority to stop work, training, and incentives that reward finding problems rather than hiding them.

Where uncertainty remains. Few controlled studies measure whether AI checklists reduce real incidents. Self-reported compliance can be inflated, long lists invite box-ticking, and automated checkers can be gamed (see Recent Developments).

Why Checklists Work, and When They Fail

Checklists rest on a few quantitative ideas about human error and layered defenses.

Small skip rates compound

If a project has n critical steps and each is skipped with probability q, independently, the chance that at least one is skipped is 1 - (1 - q)n. With 20 steps and a 5% skip rate per step, that is about 64%. Cut the skip rate to 1% with a checklist and it falls to about 18%. Checklists target omissions of steps people already know, which is where these compounding odds bite.

Working memory is small

People can hold only a handful of items in working memory at once: George Miller's classic estimate was "seven plus or minus two," and later research puts it closer to four meaningful chunks. Under time pressure and interruptions, steps drop out. A checklist moves that memory load onto paper or into a tool. The same limit explains why checklists must stay short: every added item competes for the attention the critical ones need.

Layers of defense

James Reason's "Swiss cheese" model treats each safeguard as a slice with holes; an accident happens when the holes line up. If three independent layers each miss an error with probabilities 0.2, 0.3 and 0.5, all three miss it only 0.2 × 0.3 × 0.5 = 3% of the time. That multiplication only works if the layers are independent. A checklist and a reviewer who both trust the same flawed benchmark share a blind spot, so good checklist items ask for independent evidence, such as a test set from a later time period or a review by someone outside the team.

Length versus attention

Each item costs time and attention, and its expected benefit is the chance it catches a problem multiplied by that problem's cost. An item that catches a $200,000 problem in one project out of 50 is worth about $4,000 per project; one that catches a $1,000 problem once in 5,000 projects is worth 20 cents and should be automated or dropped. This arithmetic favors a few "killer items" and explains why padded checklists decay into box-ticking.

Leakage: the most common silent failure

Data leakage lets information about the test set reach the model during training, so results look better than they will be in use. Sayash Kapoor and Arvind Narayanan of Princeton catalogued leakage types such as missing or improper train-test separation, preprocessing or feature selection done on the full dataset before splitting, duplicate records across splits, features that are proxies for the outcome, using future data to predict the past, and test data drawn from a different distribution than the one of interest. Each maps directly to a checklist item. When they corrected leakage in prominent civil war prediction studies, complex machine learning models no longer performed substantively better than decades-old logistic regression.

Goodhart's law and box-ticking

When completion rate becomes the measure, people optimize the measure: every box gets ticked, whether or not the step happened. The remedy is the same one used for other metrics: ask for evidence, spot-check it, and judge the checklist by outcomes (defects caught, incidents avoided), not by completion.

Principle What it predicts Design response
Compounding skips Many small omission risks add up to a likely failure Cover the steps that are skipped most and cost most
Working memory limits Steps drop out under pressure Externalize steps at defined pause points
Layered defenses Independent layers multiply protection Ask for independent evidence, not repeated self-assessment
Length versus attention Long lists dilute attention Five to nine killer items per pause point; automate the rest
Leakage Inflated results that collapse in use Specific leakage checks with linked evidence
Goodhart's law Completion becomes the goal Evidence, spot checks and outcome tracking

Why This Matters

The evidence from medicine is encouraging and cautionary. The WHO pilot study (Haynes and colleagues, 2009) covered 3,733 patients before and 3,955 after the checklist was introduced, with deaths falling from 1.5% to 0.8%; the authors called the improvement "most likely multifactorial." When Ontario later made surgical checklists mandatory across its hospitals, Urbach and colleagues found no significant change in mortality (adjusted odds ratio 0.91, 95% confidence interval 0.80 to 1.03) or complications (0.97, 0.90 to 1.03). As Urbach put it, "It's not as easy as a checklist." The lesson carries over to AI: the list is a tool, and training, culture and genuine use decide the outcome.

AI-specific evidence points the same way. In interviews with 53 AI practitioners in India, East and West Africa and the United States, Google researchers found that 92% had experienced "data cascades": compounding downstream failures caused by data issues that were usually avoidable. The NeurIPS 2019 reproducibility program, which combined a code policy, a community reproducibility challenge and a checklist, raised the share of accepted papers with code from under 50% the year before to 74.4% at camera-ready; yet only 34% of reviewers said the checklist answers helped them evaluate submissions. Repositories from NeurIPS 2019 papers that met all five items of the ML Code Completeness Checklist had a median of 196 GitHub stars, the most of any group.

Engineering implications. Checklists are among the cheapest forms of AI governance. They fit into pull request templates, model registry gates, data review meetings and release reviews. The most effective ones link each item to evidence and turn the mechanical items into automated tests, as the ML Test Score recommends. See Best Practices, Project Check-in and Evaluation.

Governance and regulation increasingly take checklist form. The European Commission's High-Level Expert Group on AI turned its seven requirements for trustworthy AI into a self-assessment checklist and web tool (ALTAI). U.S. federal agencies must document minimum practices for high-impact AI, including pre-deployment testing, an impact assessment, ongoing monitoring, human oversight and appeals, a list that maps directly onto a release checklist. Model cards and datasheets give buyers and auditors a standard set of questions to put to vendors. See AI Governance and Risk, Compliance and Regulation.

What would change if AI checklists were validated like surgical ones. Organizations could show that specific checks prevent specific failures, auditors and insurers could rely on them, and reporting standards would make AI studies and products comparable. What remains uncertain: whether AI checklists reduce real incidents, how to keep them short as risks multiply, and how to prevent performative compliance.

Recent Developments

Developments from roughly mid-2024 to October 2026. Each topic carries an evidence status: demonstrated means peer-reviewed or controlled results in specific settings; measured trend means counts from incident databases or surveys; consensus guidance means structured expert agreement rather than tested outcomes; preliminary means small, self-selected or early studies; proposed interpretation and hypothesis mark explanations that have not been tested directly; and open questions are listed at the end.

Reporting standards for machine-learning research

  • REFORMS. Published in Science Advances in 2024, REFORMS is a 32-item checklist in eight sections (study design, computational reproducibility, data quality, data preprocessing, modeling, data leakage, metrics and uncertainty quantification, and generalizability), built by consensus among 19 researchers from computer science, mathematics, social sciences and health research.
  • TRIPOD+AI. Published in the BMJ on 16 April 2024, TRIPOD+AI is a 27-item checklist (52 subitems) for studies that develop or evaluate clinical prediction models, whether they use regression or machine learning. It supersedes TRIPOD 2015, was developed through a two-round Delphi survey (170 and 200 respondents) and a consensus meeting, and comes with a separate 13-item checklist for abstracts.
  • Pitfalls guide. Michael Lones's guide to avoiding machine learning pitfalls, organized by stage (before building models, building reliably, evaluating robustly, comparing fairly, reporting), was published in Patterns in 2024.
  • Leakage tally. Princeton's running survey, updated in May 2024, listed 41 papers from 30 fields that had found leakage-type errors, collectively affecting 648 papers.
  • Evidence status: consensus guidance backed by documented failures; whether these checklists improve research quality has not yet been measured in controlled studies.

Conference checklists and their automation

  • Required checklists. NeurIPS 2025 required every submission to include a paper checklist on reproducibility, transparency, research ethics and societal impact, placed in the same PDF after the references and excluded from the page limit; papers that fail submission requirements may be rejected without consideration of their merits.
  • An LLM as checklist assistant. In a NeurIPS 2024 experiment, authors could submit papers to an LLM-based assistant that reviewed their checklist answers. Across 234 submissions, more than 70% of surveyed authors found it useful and said they would revise their papers, but inaccuracy (20 of 52 comments) and excessive strictness (14 of 52) were common complaints. In a red-team test, a GPT-4o "attack" agent that rewrote only the checklist justifications raised the assistant's scores on 14 of 15 questions without changing the paper, partly by inventing details such as a hardware description and a placeholder code repository URL. The organizers concluded that such a tool "should likely not be used as a fully automated review tool that replaces human review."
  • Evidence status: preliminary, from self-selected participants and survey responses; the organizers note they cannot attribute paper changes causally to the tool.

Checklists as evaluation instruments for language models

  • Background: behavioral testing. CheckList (Ribeiro, Wu, Guestrin and Singh; ACL 2020 best paper) organizes tests into a matrix of capabilities and test types. In its user study, practitioners using CheckList wrote about twice as many tests and found almost three times as many bugs as those without it.
  • Generated checklists for AI judges. TICK (Cook and colleagues at the University of Oxford, UCL and Cohere, October 2024) has a model break each instruction into yes/no checklist questions before grading. Exact agreement between the AI judge and human preferences rose from 46.4% to 52.2%, using the checklists for self-refinement improved reasoning tasks on LiveBench by 7.8 points, and giving the checklists to human annotators raised their agreement from 0.194 to 0.256.
  • Rubrics at scale. OpenAI's HealthBench (May 2025) scores responses in 5,000 realistic health conversations against 48,562 criteria written by 262 physicians who have practiced in 60 countries. A model grader checks each criterion, and the authors report it agrees with physicians about as often as physicians agree with each other. The benchmark also stresses worst-case reliability, because "a single unsafe or incorrect answer can outweigh the benefit of many good ones."
  • Checklists as training rewards. Checklists Are Better Than Reward Models For Aligning Language Models (Carnegie Mellon University and Apple; NeurIPS 2025) introduced reinforcement learning from checklist feedback (RLCF), which scores each response against instruction-specific checklist items and uses the result as a reward. Fine-tuning Qwen2.5-7B-Instruct this way improved the hard satisfaction rate on FollowBench by 4 points, InFoBench by 6 points and the Arena-Hard win rate by 3 points, and it was the only method tested that helped on every benchmark.
  • Proposed interpretation. A common explanation for these gains is decomposition: breaking a vague judgment ("is this a good answer?") into specific yes/no requirements makes grading more consistent and feedback more specific. The explanation fits the results above but has not been isolated experimentally.
  • A qualification. A 2025 study by Furuhashi, Nakayama, Kodama and Sugawara tested six ways of generating checklists across eight model sizes. Using checklists selectively tended to improve evaluation when comparing two outputs, but the benefit was less consistent for direct scoring. Even checklist items that correlated poorly with human scores often matched human-written criteria, which points to inconsistency in human grading itself and to the need for clearer criteria on both sides.
  • Evidence status: demonstrated on specific benchmarks and models; how well the gains carry over to other tasks and to high-stakes uses is still open.

Checklists for benchmarks and agents

  • Agentic Benchmark Checklist. Zhu and colleagues (NeurIPS 2025) found that SWE-bench Verified used insufficient tests and that τ-bench counted empty responses as successes, flaws that can misstate agent performance by up to 100% in relative terms, and published a checklist for building rigorous agent benchmarks.
  • Construct validity. A review of 445 language model benchmarks by 29 experts (NeurIPS 2025) found that only 16% used statistical tests when comparing models, and turned its findings into a checklist for benchmark designers.
  • Evidence status: demonstrated flaws in the benchmarks studied; the checklists themselves are consensus guidance.

Security, governance and incidents

  • Security checklists. The OWASP GenAI Security Project's LLM Applications Cybersecurity and Governance Checklist (version 1.1, May 2024) targets executives and teams in technology, cybersecurity, privacy, compliance and legal roles, and the OWASP Top 10 for LLM Applications 2025 serves as a standard threat checklist.
  • Policy as checklist. OMB memorandum M-25-21 (April 2025) requires U.S. federal agencies to document minimum practices for high-impact AI within 365 days.
  • Incidents keep rising. Stanford's 2026 AI Index reports that the AI Incident Database recorded 362 incidents in 2025, up from 233 in 2024. Organizations reporting no AI policies at all fell from 24% to 11%, yet the share rating their AI incident response as "excellent" fell from 28% to 18%: more policies on paper, less confidence in practice. Among the frontier models the report tracked, only Claude Opus 4.5 reported results on more than two of the responsible AI benchmarks it follows.
  • Evidence status: measured trend for the incident and survey figures; incident counts reflect what gets reported and covered in the media, not every incident. The OWASP and OMB items are guidance and policy, not evidence of effect.

Open questions

  • Do AI checklists measurably reduce incidents, as the WHO pilot suggested for surgery, or do they fade into paperwork, as the Ontario rollout suggests can happen?
  • Which items are the true "killer items" for machine learning, and for generative AI and agents?
  • Can LLM-based checklist reviewers be trusted when the NeurIPS experiment showed they can be gamed?
  • How should checklists keep pace with models and threats that change every few months?
  • Hypothesis: Gawande separates errors of ignorance (not knowing enough) from errors of ineptitude (knowing but failing to apply what is known), and checklists target the second kind. If that holds for AI, checklists should help most with known failure modes such as leakage, untested subgroups and missing monitoring, and least with failures nobody yet knows how to test for, such as new agent behaviors. No study has tested this directly.

Efforts aimed at these questions include REFORMS, TRIPOD+AI, the NeurIPS checklist experiments, the Agentic Benchmark Checklist and checklist-based evaluation research.

Checklists to Use

A curated set of checklists, from personal project lists to community standards. Pick the family that matches your pause point (see The Mechanism) and trim it to the items that matter for your project.

Project and data checklists

Documentation, reproducibility and reporting checklists

Responsible AI, security and governance checklists

Testing and evaluation checklists

Cheat sheets and reading lists

Common Mistakes

YouTube search... ...Google search

Mistakes that checklists catch

Most mistakes below are well known, which is exactly why a checklist helps: the risk is forgetting, not ignorance.

Mistake Typical symptom Checklist item
Data leakage Results that look too good, then collapse in use Split before any preprocessing or feature selection; check for duplicates across splits; split by time for temporal data
Unrepresentative data Accuracy drops after launch, or for particular groups Compare training data with the deployment population; evaluate by subgroup
No baseline No way to tell whether machine learning adds value Compare against a simple rule and a simple model, such as logistic regression
Tuning on the test set Optimistic results that do not replicate Keep a final test set that is used once, after all choices are made
Wrong metric or threshold The model "wins" but the business decision does not improve Tie the metric to the decision; set thresholds by the cost of each error (see Evaluation - Measures)
Default loss function taken for granted The model optimizes something nobody cares about Choose the loss to match the error costs and the data, for example robustness to outliers
Ignored outliers and data quality Unstable models and "data cascades" downstream Profile the data, document known issues, add automated validation (see Data Quality)
Cyclical features treated as linear Midnight and 11 p.m. look far apart to the model Encode hours, days and months with sine and cosine
Regularization without standardization L1 or L2 penalties hit features unevenly because of their units Standardize features before regularized fitting
Coefficients read as feature importance Misleading explanations, especially with correlated features Use dedicated importance or explanation methods and check collinearity
Specification gaming The system meets the letter of the objective but not its intent Inspect behavior, not just the score; test for loopholes in the objective
Unverified generative output Fabricated facts or citations reach users or courts Check every factual claim and citation against its source; disclose AI use
No monitoring Silent degradation as data drifts Name an owner, set drift and performance alerts, schedule re-evaluation (see model monitoring)

Load video
YouTube
Common mistakes made in Machine Learning Models
Analytics University You will learn the common mistake people make while building machine learning models. Machine learning models are easy to build but need attention to details. The common mistakes could be: 1- taking Default Loss Function for granted, 2- Using one Algorithm / Method For All Problems: 3- Ignoring Outliers: 4- No Proper Dealing With Cyclical Features, 5- L1/L2 Regularisation Without Standardization, 6- Interpreting Coefficients From Linear or Logistic Regressions as features importance. Analytics Study Pack : https://analyticuniversity.com/

Load video
YouTube
Cameron Davidson Pilon: Mistakes I've Made
PyData Seattle 2015 In this humbling talk, I'll describe some mistakes I've made in working in statistics and machine learning. I'll describe my original intentions, symptoms, how I eventually discovered the mistake, and possibly even a solution. The topics include mistakes in A/B testing, Kaggle competitions, data collection, and other fields. I'll also introduce some interesting statistical and machine learning counterexamples: examples where our original intuition fails, and solutions to these examples.

Load video
YouTube
AI Simplified: Top 3 Rookie Mistakes in Machine Learning
John Boersma, Director of Education at DataRobot, shares his list of the top three rookie mistakes in machine learning for our AI Simplified series. Learn more about simplified AI terms on our wiki page: https://www.datarobot.com/wiki

Load video
YouTube
Top 10 Machine Learning Pitfalls: Mark Landry
Over-fitting, misread data, NAs, collinear column elimination and other common issues play havoc in the day of practicing data scientist. In this talk, Mark Landry, one of the world’s leading Kagglers, will review the top 10 common pitfalls and steps to avoid them. View more talks from H2O Open Tour Dallas https://open.h2o.ai/dallas.html Powered by the open source machine learning software H2O.ai. Contributors welcome at https://github.com/h2oai To access slides on H2O open source machine learning software, go to: https://www.slideshare.net/0xdata

AI Failures

Public failures show which checklist items matter most. Trackers such as the OECD AI Incidents and Hazards Monitor and the AI Incident Database collect them; the latter recorded 362 incidents in 2025, up from 233 in 2024. In each case below, a step that belongs on a checklist could have caught the problem or limited the damage.

Failure What happened Checklist item that would have flagged it
Amazon recruiting tool (reported 2018) A résumé-screening model trained on ten years of applications, mostly from men, penalized résumés containing the word "women's" and downgraded graduates of two all-women's colleges. Amazon disbanded the team by early 2017. Audit training data for historical bias; evaluate outcomes by group before any use
Epic Sepsis Model (external validation 2021) At Michigan Medicine the model reached an AUC of 0.63, against 0.76 to 0.83 in the developer's documentation, missing 67% of sepsis cases while alerting on 18% of all hospitalized patients. Validate vendor claims on your own population before deployment; measure the alert burden
Mata v. Avianca (2023) Lawyers filed a brief citing nonexistent cases generated with ChatGPT; a federal judge imposed a $5,000 sanction, noting that existing rules "impose a gatekeeping role on attorneys to ensure the accuracy of their filings." Verify every citation against its source; never treat a chatbot as a source
Air Canada chatbot (2024 ruling) A tribunal held the airline liable for its chatbot's wrong answer about bereavement fares. Ground answers in current policy; test the most common customer questions; route uncertain cases to people
Replit coding agent (July 2025) An AI agent deleted a production database during an explicit code freeze. Least-privilege credentials; separate development and production; human approval for destructive actions
Deloitte Australia report (2025) An A$440,000 government report contained a fabricated court quote and references to research that does not exist; Deloitte refunded the final payment. Check every quote and reference; disclose AI use; require reviewer sign-off
Specification gaming (DeepMind catalog, 2020) A boat-racing agent circled to hit the same reward targets instead of finishing the race; a block-stacking agent flipped a block instead of stacking it. Compare the objective with the intended outcome; watch behavior, not just the reward

Load video
YouTube
"Machine learning failures - for art!" by Janelle Shane
It's tough to write a machine learning algorithm that works well. Overfitting, noisy data, a problem that's too general: these problems plague the programmers who apply these algorithms to financial modeling and image labeling. But mistakes can also be fun. At her humor blog AIweirdness.com, Janelle Shane posts examples of machine learning algorithms going terribly, hilariously wrong. Here, she talks about some common machine learning mistakes, and how to use them deliberately.

Load video
YouTube
Lessons Learned from Machine Learning Gone Wrong - Janelle Shane
A related talk by Shane on the same theme: what machine learning failures reveal about overfitting, noisy data and problems that are too general. About Janelle: Janelle Shane trains neural networks, a type of machine learning algorithm, to write unintentional humor as they struggle to imitate human datasets. Well, she intends the humor. The neural networks are just doing their best to understand what's going on. Currently located on the occupied land of the Arapahoe Nation.

Sources and Further Viewing

Additional Viewing

Load video
YouTube

Universal Medical Intelligence: OpenAI's Plan to Elevate Human Health, with Karan Singhal (The Cognitive Revolution podcast, February 2026, about two hours). Singhal of OpenAI explains how HealthBench grades medical conversations against roughly 49,000 physician-written criteria, why the team built a consensus subset and a "Hard" subset, how a model-based grader compared with physicians, and why it tracks worst-case performance across repeated samples. It shows what a checklist looks like when it is scaled into a full evaluation system. Watch on YouTube

Load video
YouTube

Why AI evals are the hottest new skill for product builders (Lenny's Podcast with Hamel Husain and Shreya Shankar, September 2025, 1 hour 46 minutes). Husain and Shankar show how to turn error analysis of real transcripts into a set of binary pass or fail criteria, one per failure mode, and how to validate AI judges against human labels. It shows how a team builds its own evaluation checklist from evidence instead of copying a generic one. Watch on YouTube

Load video
YouTube

The Reproducibility Crisis in ML-based Science (Princeton online workshop, 28 July 2022, full-day recording). Organized by Sayash Kapoor, Arvind Narayanan, Priyanka Nanayakkara, Kenny Peng and Hien Pham, the workshop brought ten speakers together on leakage and reproducibility failures across scientific fields. It is the background to the leakage taxonomy and the REFORMS checklist described above. Watch on YouTube

Load video
YouTube

Lecture 7: Troubleshooting Deep Neural Networks (Josh Tobin, Full Stack Deep Learning, Spring 2021 course). Tobin turns debugging into a five-step checklist: start simple, implement and debug (overfit a single batch and reproduce a known result), evaluate with a bias-variance decomposition, improve the model or data, and tune hyperparameters with coarse-to-fine random search. It is a practical read-do checklist for a task that often goes wrong. Watch on YouTube

Key Sources

Origins and evidence from other fields

Evidence from AI practice

Checklist-based evaluation and training

Incidents and failures

Retrieved from "https://primo.ai/w/index.php?title=Checklists&oldid=46779"