Mathematics

From PRIMO.ai
Jump to navigation Jump to search

YouTube ... Quora ...Google search ...Google News ...Bing News


Mathematics is the working toolkit underneath artificial intelligence. Every modern AI system, from a spam filter to a Large Language Model (LLM), is built from a small number of mathematical ideas used at enormous scale: linear algebra to store and transform data, calculus to learn from mistakes, probability to express uncertainty, statistics to tell real progress from noise, optimization to search for good parameters, and Fourier and signal analysis to move between time, space, and frequency. Logic and discrete mathematics add the machinery of exact reasoning and proof. This page has two focuses. The first explains those ideas and collects the best videos, courses, and books for learning them; the second, which became urgent in 2026, examines the mathematics that AI has started to give back to the world and what that means for how mathematics is practiced, published, and taught.

The ideas have many architects. Legendre (1805) and Gauss (1809) gave us least squares, the ancestor of every loss function. Joseph Fourier showed in the early 1800s that complicated signals are sums of simple waves. Cauchy described gradient descent in 1847. Rosenblatt's perceptron (1958) turned a weighted sum into a learning machine, and Rumelhart, Hinton, and Williams popularized backpropagation in 1986, which is just the chain rule applied carefully. Volker Strassen proved in 1969 that matrices can be multiplied faster than the schoolbook method, and Vaswani and colleagues at Google introduced the Transformer in 2017, an architecture that is mostly matrix multiplication plus a softmax. Today the frontier is shaped by research groups at Google DeepMind (AlphaTensor, AlphaProof, AlphaEvolve), OpenAI, Anthropic, Microsoft Research, and several Chinese labs; by independent evaluators such as Epoch AI; by the Lean proof-assistant community; and by mathematicians such as Terence Tao and Thomas Bloom who track what AI actually contributes. Educators like Gilbert Strang (MIT) and Grant Sanderson (3Blue1Brown) have done more than anyone to make the math teachable.

The problem this math solves is simple to state: turn data into reliable predictions, prove that the predictions are reliable, and do it within a budget of computation and energy. The central mechanism is just as compact. Represent everything as vectors. Transform the vectors with matrices. Turn the results into probabilities. Measure the error. Use calculus to nudge millions or billions of parameters in the direction that shrinks the error. Repeat.

The paradigm shift is that intelligence stopped being hand-written as rules and started being learned as functions. Classical programming encodes knowledge explicitly; machine learning encodes it implicitly in the numbers of weight matrices found by optimization. Between 2024 and 2026 a second shift arrived: AI went from being a consumer of mathematics to a producer of it. AI systems earned perfect scores at the International Mathematical Olympiad, saturated research-level benchmarks, contributed to resolving open Erdős problems, and helped find a counterexample to the 87-year-old Jacobian conjecture. Then, in September and October 2026, the pace turned into what many mathematicians called a crisis: Claude agents built a 13-million-line machine-checked proof of Fermat's Last Theorem, OpenAI claimed a Lean-verified answer to a forced version of the Navier-Stokes Millennium Prize Problem, a Claude-discovered algorithm overturned two central hypotheses of fine-grained complexity theory, and an OpenAI release of 722 manuscripts claimed the quasi-Riemann hypothesis, the unique games conjecture, and a matrix multiplication exponent of at most 9/4. Many of these claims are still being checked, and the section What AI Has Given Mathematics separates what is verified from what is only asserted. AI systems have also started improving the mathematics that runs AI itself, for example by discovering faster matrix multiplication procedures and tightening the theoretical bound on how fast matrix multiplication can ever be. That feedback loop is why this page now pays as much attention to verification, statistics, and the health of the mathematical community as to formulas.


There are three kinds of lies: lies, damned lies, and statistics.
Popularized by Mark Twain, who credited Benjamin Disraeli...
The warning applies to every benchmark score and every claimed breakthrough on this page.


The Mechanism

Here is how the math turns data into a working model, step by step. Each step names the branch of mathematics doing the work, so the list doubles as a study plan. The first seven steps describe essentially every neural network since the 1980s, and the eighth is the layer that reasoning models added after 2024.

  1. Represent (linear algebra). Words, pixels, and sensor readings become vectors of numbers called embeddings. A batch of sentences becomes a tensor with a shape such as (batch, sequence length, model width). Similarity between two items is measured with a dot product: a·b = Σ aibi = ‖a‖ ‖b‖ cos θ.
  2. Transform (linear algebra plus nonlinearity). A layer computes y = f(Wx + b), where W is a matrix of learned weights, b is a bias vector, and f is a nonlinear activation function. The nonlinearity matters: without it, stacking layers would collapse into a single matrix, because a product of matrices is still just a matrix.
  3. Mix context (attention). Transformers let every token look at every other token through attention: Attention(Q, K, V) = softmax(QKT / √dk) V. Q, K, and V are the input multiplied by three learned matrices. The QKT grid has one entry per pair of tokens, so its cost grows with the square of the sequence length.
  4. Predict (probability). The softmax function turns raw scores zi into probabilities pi = ezi / Σj ezj. The model's output is a probability distribution over possible next tokens, classes, or actions.
  5. Score the mistake (probability and information theory). Cross-entropy loss, L = -log pcorrect, charges the model for putting low probability on the right answer. Minimizing it is the same as maximum-likelihood estimation, and it is measured in bits or nats of "surprise."
  6. Learn (calculus). The chain rule, organized as backpropagation, computes the gradient ∇θL: how much the loss changes as each parameter changes. Gradient descent then updates every parameter: θ ← θ - η∇θL, where η is the learning rate. Practical optimizers such as Adam add momentum and per-parameter step sizes.
  7. Check (statistics). The trained model is tested on data it never saw. Held-out accuracy, the confusion matrix, confidence intervals, and significance tests decide whether an improvement is real or noise.
  8. Search and verify (the newer layer). Reasoning models generate long chains of intermediate steps (Chain of Thought (CoT)) and are trained with reinforcement learning on problems whose answers can be checked automatically. Systems such as AlphaEvolve wrap a language model in an evolutionary loop: the model proposes programs, automated evaluators score them, and the best survive. For proofs, proof assistants such as Lean check every logical step mechanically.

Dependencies and conditions. The mechanism only works when the training data resemble the situations the model will face (see Out-of-Distribution (OOD) Generalization), when there is enough compute to train at scale, when number formats keep enough precision for gradients to remain meaningful, and, for the search-and-verify layer, when a trustworthy checker exists. When any of these conditions fails, the same mathematics still runs, but its outputs stop deserving trust. Mathematics is unusual because answers can often be verified exactly, which is a large part of why AI progress in math has been so fast.

Where uncertainty remains. There is still no complete theory of why heavily overparameterized networks generalize instead of memorizing. Scaling laws, which predict loss from model size and data, are empirical fits rather than derived results. Researchers also disagree about how much of a model's mathematical skill is genuine algorithmic reasoning and how much is brittle pattern matching; both kinds of evidence are covered in Mathematical Reasoning and Did AI Expand Mathematics or Just Connect the Dots?.

Why This Matters

  • Scientific importance. Mathematics is both the language AI is written in and, since 2025, a domain where AI contributes new results. When machines can produce plausible proofs faster than humans can read them, the bottleneck in mathematics moves from discovery to verification and then, as the autumn 2026 releases showed, to digestion: understanding, explaining, and building on what has been proved.
  • Engineering implications. Training and running frontier models costs gigawatt-scale data centers. Because matrix multiplication dominates the workload, a few percent saved on one kernel, one bit removed from a weight, or one multiply replaced by an add compounds across trillions of operations. AlphaEvolve's 23 percent speedup of one Gemini matrix-multiplication kernel cut total training time by about 1 percent, which at that scale is a meaningful amount of energy and money. By contrast, OpenAI's claimed bound ω ≤ 9/4 transforms the theory of fast linear algebra without speeding up today's chips, because the algorithms behind such bounds only win for astronomically large matrices.
  • Practical consequences for practitioners. Reading a model card or benchmark chart now requires statistical literacy. A difference of five points on a 43-problem benchmark is usually not significant, and a benchmark built with funding from a lab that also has privileged access to part of it needs that disclosed. The Statistics in AI Evaluation section shows the arithmetic.
  • Potential applications. Faster linear algebra benefits scientific simulation, graphics, cryptography, and signal processing as much as AI. The October 2026 refutation of the 3SUM and all-pairs shortest paths hypotheses immediately produced faster algorithms for many related problems, and AI-assisted mathematics promises help with literature search, routine lemmas, counterexample search, and formalization, freeing people for the conceptual parts of research.
  • Conceptual implications. Formal proof checking separates trust in a result from trust in whoever, or whatever, produced it. A Lean-verified proof is equally valid whether a student, a professor, or a model wrote it, provided the statement was formalized correctly. The 2026 formalizations of Fermat's Last Theorem and of OpenAI's Navier-Stokes claim show this separation operating at a scale no human team could check by hand.
  • What would change if current trends hold. Terence Tao has described a coming era of "industrial-scale" mathematics, with large human and machine teams working on many problems at once, and in October 2026 he proposed a "Math 2.0" that rewards exposition and new directions rather than only first solutions. AI-designed algorithms would routinely improve the chips and kernels that train the next AI, tightening the feedback loop.
  • Limitations and uncertainty. Before September 2026, most AI successes on open problems were constructions, counterexamples, or problems that were open mainly because few people had looked at them. The autumn 2026 releases changed that picture but created new problems: manuscripts experts struggle to read, results that no human author stands behind, formalizations that sometimes prove weaker statements than the papers claim, and a verification process that lags far behind generation. Costs remain high, too: Epoch AI reports that solving five significant Erdős problems took over $220,000 of compute across hundreds of attempts, and OpenAI's Navier-Stokes run used about 10,000 agents. The mathematical community, through the 2026 Leiden Declaration and later statements, has warned about attribution, peer review, and the influence of commercial interests on which questions get asked.

Recent Developments

The period from mid-2025 to October 2026 changed this subject more than the preceding decade. Two threads run in parallel: AI doing mathematics, and mathematics making AI cheaper. The table separates what has been demonstrated from what remains preliminary, and the autumn 2026 results are examined in depth in What AI Has Given Mathematics.

Date Development What was shown Evidence status
May 2025 AlphaEvolve (Google DeepMind) 4×4 complex matrices multiplied with 48 scalar multiplications, beating a 56-year-old record of 49 Demonstrated; algorithm published and checkable
Jul 2025 Gemini Deep Think at IMO 2025 35 of 42 points with natural-language proofs inside contest time limits Officially graded gold; compute not disclosed
Nov 2025 AlphaProof in Nature Reinforcement learning in Lean on about 80 million auto-formalized statements Peer reviewed; competition-level problems only
Dec 2025 to Jan 2026 Aletheia on the Erdős database Many problems marked open were open because of obscurity, not difficulty Preprint with a careful taxonomy of results
Feb and Jun 2026 First Proof challenge Best systems answered roughly 6 to 7 of 10 research lemmas essentially correctly Expert graded; small samples
May 2026 Erdős unit distance conjecture Polynomial improvement over the square-grid construction, disproving an 80-year-old conjecture Checked by external mathematicians; later formalized in Lean
Jul 2026 Jacobian conjecture counterexample Explicit degree-7 map of 3-dimensional complex space with constant Jacobian determinant -2 that is not injective Computer-checkable; dimension 2 still open; peer review pending at time of reporting
Jul 2026 IMO 2026 Two systems scored a perfect 42 of 42 under official grading Officially graded; self-reported results from other labs are harder to audit
Aug 2026 Matrix multiplication exponent Upper bound improved from ω < 2.371339 to ω < 2.371177 with AlphaEvolve-refined optimization Preprint; small but genuine improvement
Sep 2026 Fermat's Last Theorem formalized (Anthropic) About 13 million lines of Lean written by Claude agents in 11 days, checked down to Lean's three standard axioms Machine-checked and compiled by Kevin Buzzard; formalizes Wiles's known theorem
Sep 2026 Navier-Stokes blow-up claim (OpenAI) Finite-time singularity for the 3D equations driven by a smooth external force (Clay statements C and D), with a Lean formalization Lean-checked; not accepted by the Clay Institute; unforced case still open
Sep 2026 FrontierMath Tier 4 saturated Top score 97.6 percent; every problem solved at least once Independent evaluator; benchmark funded by OpenAI
Sep 2026 FrontierMath Erdős Best model solved 2 of 68 significant open problems with Lean-verified proofs under a fixed budget Fully transparent protocol; small numbers
Oct 2026 3SUM and APSP hypotheses refuted First polynomial speedups over the textbook algorithms, found by an internal Anthropic model and written up by Alman and Vassilevska Williams Preprint with Lean formalization; simplified and improved by others within days
Oct 2026 OpenAI release of 722 manuscripts Claims include the quasi-Riemann hypothesis, the unique games conjecture, RL = L, and ω ≤ 9/4 About half Lean-certified; three withdrawn and 14 revised within days; expert review ongoing

From Olympiad Medals to Perfect Scores

In 2024, Google DeepMind's AlphaProof and AlphaGeometry 2 reached a silver-medal score at the IMO, but only after two to three days of computation and with problems hand-translated into the Lean proof language. AlphaProof's methods were published in Nature in November 2025: it learned by reinforcement learning, AlphaZero-style, inside Lean, training on millions of auto-formalized problems, and for the hardest problems it generated and learned from millions of related variants at test time. At IMO 2025 the approach changed. An advanced version of Gemini with Deep Think earned an officially graded gold medal (35 of 42 points, five of six problems) writing ordinary natural-language proofs within the 4.5-hour sessions. OpenAI reported the same score from an experimental reasoning model graded by former medalists rather than through the official process, and in November 2025 DeepSeek released DeepSeek-Math-V2, described as the first open-weight model to reach gold-medal level.

At IMO 2026 in Shanghai, Huawei's "Celia" and Xiaohongshu's "dots-note 3.0" both received perfect scores of 42 of 42 under the IMO's official judging process; only seven of the 666 human contestants did the same. Several other labs, including OpenAI, Anthropic, Axiom Math, and Moonshot AI, reported perfect scores in self-administered evaluations, some graded by AI judges. Terence Tao has repeatedly cautioned that results depend heavily on the testing setup, and IMO officials have said they cannot independently validate compute budgets, human involvement, or reproducibility for self-reported entries. What this establishes: curated competition problems, with known solutions of bounded length, are now within reach of AI systems. What it does not establish: that the same systems can do research, where problems may have no known solution, may require new definitions, or may be stated incorrectly.

Research Benchmarks and the Saturation Problem

Epoch AI launched FrontierMath in November 2024 with about 300 original, unpublished problems written by expert mathematicians, graded automatically, and designed to resist contamination. When Tier 4, built largely by professors and postdocs from their own research, launched in July 2025, the best models solved roughly 5 percent. In June 2026, Epoch released version 2 after an audit found errors in 42 percent of problems; it corrected or removed them, leaving 338 problems (295 in Tiers 1 to 3 and 43 in Tier 4). In September 2026, OpenAI's GPT-6 Astra scored 97.6 percent on Tier 4 and solved the last problem no AI had cracked, so Epoch declared Tier 4 saturated, meaning every problem has been solved at least once across models and attempts. The next-best models scored in roughly the 83 to 90 percent range.

Two caveats matter. First, FrontierMath was developed with funding from OpenAI, which has exclusive access to a subset of the problems; Epoch discloses this on every results page. Second, the v2 corrections show that even carefully built benchmarks can contain many flawed problems, which inflates noise in every comparison made before the fix.

Epoch's response was FrontierMath Erdős, announced in September 2026: 68 open problems that Thomas Bloom, who maintains the erdosproblems.com database, judged both significant and difficult. A solution counts only if the model produces a proof or disproof that passes a cheat-resistant Lean checker within a $300, 72-hour budget per problem. In the benchmark run, only GPT-6 Astra solved anything, disproving problem 74 and proving problem 126 (3 percent). In additional, non-benchmark attempts with larger budgets it solved five of the 68 at least once, at a total compute cost above $220,000. Epoch's summary was a useful calibration point at the time: AI-driven breakthroughs on significant problems were real but still uncommon, a picture the October 2026 releases would challenge within weeks.

Erdős Problems: From Literature Search to Open Conjectures

Paul Erdős posed many hundreds of problems; Bloom's database lists about 1,200, of which roughly 650 were unsolved as of September 2026. They became an informal AI benchmark in late 2025, and the early history is a cautionary tale. In October 2025, a widely shared claim that GPT-5 had "solved" ten open Erdős problems was withdrawn when it turned out the model had found existing solutions in the literature. That literature search was genuinely useful, but it was not new mathematics. The Aletheia study (December 2025) applied an agent built on Gemini Deep Think to every problem then marked open and concluded that the open status of those it resolved reflected obscurity more than difficulty. It also warned that several problem statements did not match what Erdős likely intended.

By January 2026, results became harder to dismiss. Tao described a solution to problem #728, produced with GPT-5.2 Pro and the Aristotle formalization system, as near-autonomous and not found in prior literature. According to Tao's wiki, AI tools had helped move about 100 problems into the solved column between October 2025 and February 2026, most of them minor.

The first result on a famous problem came on May 20, 2026. In the planar unit distance problem (Erdős #90), you place n points in the plane and count pairs exactly one unit apart. Erdős's square-grid construction gives about n1 + c/log log n such pairs, and he conjectured that nothing does better than n1 + o(1). An internal OpenAI reasoning model produced an infinite family of configurations giving n1 + δ pairs for a fixed δ > 0, a polynomial improvement that disproves the conjecture. Princeton mathematician Will Sawin made the constant explicit (δ ≥ 0.014). External mathematicians checked the argument, and a later Lean formalization ran to about 1.2 million lines because it had to rebuild deep number theory not yet in Lean's library, compared with an 18-page natural-language paper. OpenAI has not disclosed all procedural details.

First Proof: Research Lemmas Under Controlled Conditions

Frustrated by opaque industry claims, eleven research mathematicians from institutions including Harvard, Stanford, Yale, Columbia, and EPFL created First Proof: ten unpublished lemmas that arose naturally in their own research, released February 5, 2026, with a one-week deadline. Unrestricted public models confidently produced proofs for all ten in the pilot, but expert review found most of them wrong. Google DeepMind reported that its Aletheia agent solved six of ten by majority expert assessment, with experts split on one. In June 2026 the First Proof team ran a second batch itself on publicly available models and had about 30 mathematicians grade the answers. The best model got six to seven of ten essentially right. The lesson: AI is now useful for research-level lemmas, but confident wrong answers remain common, and grading them is, in one organizer's words, a painful and thankless task.

The Jacobian Conjecture

The Jacobian conjecture, posed by Ott-Heinrich Keller in 1939 and listed by Stephen Smale among the most important problems for the 21st century, says that a polynomial map F from n-dimensional complex space to itself whose Jacobian determinant is a nonzero constant must have a polynomial inverse. (The determinant section explains what the Jacobian determinant measures.) On July 20, 2026, mathematician Levent Alpöge, working with Anthropic's Claude Fable 5 on a problem suggested by Akhil Mathew, posted an explicit degree-7 polynomial map from three-dimensional complex space to itself with Jacobian determinant identically equal to -2 that sends three distinct points to the same point. A map that is not one-to-one cannot have an inverse, so the conjecture fails in dimension three, and by adding extra coordinates, in every higher dimension. The two-dimensional case remains open.

What makes the result unusually trustworthy is its form: a short formula whose determinant and collisions anyone can check with a computer algebra system such as Wolfram's Mathematica in seconds. What remains less clear is the division of labor between human and model, since no complete transcript of the search was published; the most defensible description is AI-assisted. Peer review was still pending in the first reports.

Community Response and Formal Verification

Published June 2, 2026, after a September 2025 workshop at the Lorentz Center in Leiden, the Leiden Declaration does not reject AI. It asks researchers to disclose AI use, keep results independently verifiable, and keep responsibility for correctness and attribution with human authors, and it warns that commercial priorities could steer which questions mathematicians pursue. Thousands of mathematicians signed it, and the International Mathematical Union endorsed it. At the International Congress of Mathematicians in Philadelphia, Tao's public lecture treated AI capability as a conjecture to be tested and argued that the field must protect goals such as explanation, mentoring, and refereeing that go beyond producing proofs. Meanwhile, Lean has become shared infrastructure: AlphaProof trains in it, FrontierMath Erdős grades in it, and AI-produced natural-language proofs are increasingly formalized after the fact. Within three months the declaration's concerns became concrete, as described in How AI Is Reframing Mathematics and Its Institutions.

Faster and Cheaper Matrix Arithmetic

The details are in Matrix Multiplication Algorithm and Alternative Matrix Algorithms. In brief: AlphaEvolve (May 2025) found a rank-48 tensor decomposition for multiplying 4×4 complex matrices, the first improvement over Strassen-based methods in that setting since 1969, and other researchers published a real-valued rank-48 scheme the following month. 2026 hardware studies found that such schemes save circuit resources only for wide operands, a reminder that fewer multiplications is not automatically faster silicon. In August 2026, a Google DeepMind team co-authored with Josh Alman and Virginia Vassilevska Williams, whose papers had set the previous records, lowered the upper bound on the matrix multiplication exponent to ω < 2.371177 by reformulating the laser-method optimization at a larger scale, designing a new optimizer, and refining it with AlphaEvolve. Two months later, OpenAI's October release claimed a far larger jump, ω ≤ 9/4, backed by a Lean formalization that independent readers have rebuilt. On the precision side, Microsoft's BitNet b1.58 2B4T (April 2025) showed that a natively ternary model can match similarly sized full-precision models, and NVIDIA's NVFP4 recipe (September 2025) showed stable 4-bit pretraining at multi-trillion-token scale. In hardware, a 2025 Nature Computational Science design stores the attention cache in volatile gain cells and computes dot products in analog form, estimating up to five orders of magnitude lower attention energy and two orders lower latency than GPUs at GPT-2-level accuracy. Those figures are architecture-level estimates, not measurements of a fabricated full system.

Looking Inside: How Models Do Arithmetic

Kantamneni and Tegmark reverse-engineered addition in GPT-J, Pythia-6.9B, and Llama 3.1 8B and found that integers are represented on a generalized helix: a linear component plus circles with periods such as 2, 5, 10, and 100. To compute a + b, the models appear to manipulate the helices for a and b into the helix for a + b, a version of the "Clock" algorithm first seen in tiny networks trained on modular arithmetic, and the authors confirmed the mechanism with causal interventions. Earlier work by Zhou and colleagues (2024) found Fourier features doing similar work in fine-tuned models. Other researchers, notably Nikankin and colleagues, argue that models solve arithmetic with a "bag of heuristics" rather than a clean algorithm, and Apple's GSM-Symbolic study (2024) showed that accuracy on grade-school word problems dropped when names, numbers, or irrelevant clauses were changed. Both pictures can be partly true: structured mechanisms exist, and so does brittleness. This is an active area of Explainable AI.

Measuring Progress Honestly

Evan Miller's paper argues that evals are experiments and should be reported like experiments: with standard errors, with clustered errors when questions come in related groups, and with paired comparisons when two models answer the same questions. Its recommendations have become more urgent as benchmarks shrink to a few dozen expert problems. See the worked examples in Statistics in AI Evaluation.

Open Questions

  • How much of AI's success on open problems reflects new ideas versus faster search through constructions that humans never tried? This is the subject of Did AI Expand Mathematics or Just Connect the Dots?
  • Can natural-language proofs be verified at the rate AI can produce them, or will formalization become mandatory for AI-assisted results?
  • Will benchmarks keep pace? FrontierMath Tier 4 went from about 5 percent to saturation in about fourteen months.
  • How should credit be shared when a human frames a problem and a model finds the key object?
  • Will the unforced Navier-Stokes problem (Clay statements A and B) fall, and can quasi-Riemann methods be pushed toward the critical line at 1/2? Experts say the method behind the 7/8 result cannot reach 1/2 without a new idea.
  • Will ternary, 4-bit, MatMul-free, and analog approaches hold their accuracy at frontier scale, and will hardware vendors build native support?
  • Is ω equal to 2? Most researchers suspect so. Progress came in steps of a few ten-thousandths until OpenAI's October 2026 claim of ω ≤ 9/4, which, if it survives expert review, is the largest single improvement in decades.

Additional Viewing

Load video
YouTube

AI Crisis in Math: Navier-Stokes, Riemann Hypothesis & Complexity Theory | Lex Fridman Podcast #503
Lex Fridman's self-described emergency episode, released October 26, 2026 and running more than three hours, brings together MIT theoretical computer scientist Ankur Moitra, University of Washington fluid dynamicist Steven Brunton, and Rutgers number theorist Alex Kontorovich to interpret the autumn 2026 results on Navier-Stokes, the quasi-Riemann hypothesis, the unique games conjecture, 3SUM, and the Fermat's Last Theorem formalization. It is the best single companion to Did AI Expand Mathematics or Just Connect the Dots? and How AI Is Reframing Mathematics, covering the convex hull versus semigroup debate, the "doorman fallacy," and how journals, Lean, and graduate training must change.
Watch on YouTube ... Transcript

Load video
YouTube

Terence Tao: "Mathematics in the Age of AI" (ICM 2026)
Terence Tao's public lecture at the 2026 International Congress of Mathematicians in Philadelphia (posted by Alvaro Lozano-Robledo). Tao treats AI's mathematical ability as a conjecture to be examined, asks what the real goals of mathematics are beyond producing proofs, including explanation, communication, mentoring, and refereeing, and discusses what changes when proofs become abundant rather than scarce. It is the best single companion to the Recent Developments section above and anticipates the "Math 2.0" ideas he developed after the October release.
Watch on YouTube

Load video
YouTube

The Riemann Hypothesis, Explained | Quanta Magazine
In this 16-minute explainer from 2021, Rutgers mathematician Alex Kontorovich walks through the prime numbers, Gauss's prime-counting function, Euler's zeta function, Riemann's extension of it to the complex plane, and why the location of the zeros controls how primes are distributed. It is short but unusually clear, and it supplies exactly the background needed for The Quasi-Riemann Hypothesis, including the critical strip that OpenAI's claimed proof narrows.
Watch on YouTube

Load video
YouTube

The Potential for AI in Science and Mathematics | Terence Tao | Oxford Mathematics
An Oxford Mathematics London Public Lecture from July 2024, followed by a conversation with mathematician Po-Shen Loh. Tao surveys AI's role in science and argues that pairing AI with proof assistants such as Lean could enable large collaborative projects with guarantees of correctness. Watching it alongside the 2026 lecture shows how quickly the field moved from possibility to practice.
Watch on YouTube

Load video
YouTube

Google AlphaEvolve: Discovering New Science (exclusive interview) | Machine Learning Street Talk
Recorded on AlphaEvolve's release day in May 2025, this long-form interview with Google DeepMind researchers Matej Balog and Alexander Novikov explains how Gemini proposes code, automated evaluators score it, and an evolutionary loop keeps the best ideas. They walk through how the system found the 48-multiplication algorithm for 4×4 complex matrices by evolving a search procedure for tensor decompositions rather than the algorithm itself.
Watch on YouTube

Load video
YouTube

Attention in transformers, step-by-step | Deep Learning Chapter 6 | 3Blue1Brown
Grant Sanderson's 26-minute visual walkthrough of queries, keys, values, the softmax attention pattern, masking, and multi-head attention, with a full parameter count for GPT-3. It shows exactly where matrix multiplication and the dot product live inside a large language model and why attention cost grows with the square of the context length.
Watch on YouTube

What AI Has Given Mathematics: Discoveries and Proofs

For most of history, mathematics flowed one way through AI: mathematicians supplied the linear algebra, calculus, and statistics, and engineers turned them into learning machines. Between 2021 and 2026 the flow reversed, slowly at first and then all at once. Early systems found patterns for humans to prove, later systems searched for constructions that beat records, and by autumn 2026 AI systems were producing complete proofs of famous conjectures, machine-checked formalizations of landmark theorems, and algorithms that overturned the working assumptions of entire subfields. This section catalogs those contributions, explains each major result in plain language, and keeps a careful line between what has been verified and what has only been claimed.

Earlier Milestones: AI as an Intuition Pump (2021 to 2025)

The first wave treated AI as a microscope for patterns rather than a prover. In 2021, DeepMind researchers working with mathematicians, including Marc Lackenby and András Juhász in knot theory and Geordie Williamson in representation theory, trained neural networks to predict one mathematical quantity from others and then used attribution techniques to see which inputs mattered. That pointed the mathematicians to a new relationship between algebraic and geometric invariants of knots and to a candidate approach for the combinatorial invariance conjecture for Kazhdan-Lusztig polynomials, results the humans then proved and published in Nature. The machine did not prove anything; it told experienced mathematicians where to look, which is exactly the role intuition usually plays.

The second wave paired language models with automatic evaluators. Google DeepMind's FunSearch (2023) asked an LLM to write and evolve short programs that construct cap sets, large collections of points in which no three lie on a line, and found constructions beyond the best known, including the largest improvement in two decades to an asymptotic lower bound. Because FunSearch outputs programs rather than raw answers, the discoveries were easy for humans to read and verify. AlphaTensor (2022) and AlphaEvolve (2025) applied similar search to matrix multiplication and dozens of other problems; AlphaEvolve also raised the known lower bound for the kissing number in 11 dimensions from 592 to 593 spheres, and in late 2025 Terence Tao and collaborators tested it on 67 problems across analysis, geometry, and combinatorics. In fluid dynamics, Google DeepMind and academic collaborators used physics-informed neural networks with computer-assisted proof techniques to discover new families of unstable self-similar singularities in equations related to the Euler equations, an early sign of the Navier-Stokes work that followed.

By mid-2026 the line between assistant and author was blurring. OpenAI's reasoning models disproved the Erdős unit distance conjecture in May and, according to OpenAI, proved the cycle double cover conjecture from graph theory in July, a result mathematicians quickly rewrote as an exposition suitable for advanced undergraduates. Google DeepMind's AlphaProof Nexus, which runs several Gemini-powered prover agents against strict Lean checkers, resolved 9 of 353 open Erdős problems and 44 of 492 conjectures from the Online Encyclopedia of Integer Sequences, although critics found that some of the "open" Erdős items had earlier human solutions. The Jacobian conjecture counterexample in July showed a human and a model working as a team. The common pattern across all of these was that humans framed the problems, machines searched enormous spaces, and checkers, whether code, Lean, or expert readers, decided what counted.

The Autumn 2026 Results

Four announcements in six weeks changed the conversation. On September 4, 2026, Anthropic reported that Claude agents had produced the first complete machine-checked proof of Fermat's Last Theorem. On September 8, OpenAI claimed a Lean-verified solution to a forced version of the Navier-Stokes Millennium Prize Problem. On October 5, Josh Alman and Virginia Vassilevska Williams posted the first polynomial speedups for 3SUM and all-pairs shortest paths, crediting the core algorithm to an internal Anthropic model. On October 6, OpenAI released 722 manuscripts, organized into 372 families of results, produced by an unreleased internal model that had been tried on about 4,000 problems and spent an average of about three hours on each result it kept. About half of the manuscripts came with Lean certificates, OpenAI's own researchers had not read all of them at release, and three were withdrawn and 14 revised within days. Lex Fridman opened his podcast on the episode by calling it an emergency.

Result Field What is claimed Source Status in late October 2026
Quasi-Riemann hypothesis Analytic number theory No zeros of the Riemann zeta function or any Dirichlet L-function with real part greater than 7/8 (a second proof gives 11/12) OpenAI, the only human-edited manuscript in the release 7/8 theorem formalized in Lean; experts have unpacked the shorter proof; independent checks under way
Exclusion of Landau-Siegel zeros Analytic number theory No exceptional "Siegel zeros" for Dirichlet L-functions OpenAI Main result formalized in Lean; an independent direct proof with explicit constants has appeared
Navier-Stokes blow-up Partial differential equations A smooth fluid at rest, pushed by a smooth compactly supported force, develops infinite velocity in finite time while its energy stays finite (Clay statements C and D) OpenAI, September 8 Lean-formalized; Clay Institute has not accepted it; the unforced statements A and B remain open
Unique games conjecture Computational complexity A deterministic polynomial-time reduction from 3SAT to Unique Games, proving Subhash Khot's 2002 conjecture OpenAI Lean-formalized; experts describe the manuscript as nearly unreadable; review ongoing
RL = L Computational complexity Randomness gives no extra power to logarithmic-space computation OpenAI Claimed; under review
Matrix multiplication exponent Algebraic complexity ω ≤ 9/4 over the complex numbers, down from 2.371177 OpenAI Lean formalization independently rebuilt; elementary human exposition with explicit algorithms published October 27
3SUM and APSP Fine-grained complexity 3SUM in about n1.9992 time and APSP in about n2.9995, refuting two central hypotheses Anthropic model; written up by Alman and Vassilevska Williams Lean formalization; simplified and improved by other researchers within days
Fermat's Last Theorem in Lean Formal mathematics A complete machine-checked proof: about 13 million lines and 30,300 theorems, resting only on Lean's three standard axioms Anthropic Compiled and reviewed by Kevin Buzzard; formalizes a known proof rather than a new one
Other October claims Many fields Partial results on the Birch and Swinnerton-Dyer and Hodge conjectures, the Hilbert-Smith conjecture, Hilbert's tenth problem over the rationals, a disproof of Hadwiger's conjecture, and integer multiplication faster than n log n on multitape Turing machines OpenAI Largely unreviewed; some unformalized; some formalizations prove weaker statements than the papers claim

The Quasi-Riemann Hypothesis

The Riemann hypothesis concerns the zeros of the zeta function, a function of complex numbers that encodes the prime numbers the way a frequency spectrum encodes a sound, which is why Alex Kontorovich describes Riemann's insight as doing signal processing on the primes (see the Fourier section). All the interesting zeros lie in a vertical "critical strip" between real parts 0 and 1, and the hypothesis says they all sit exactly on the center line at 1/2. The prime number theorem is equivalent to knowing there are no zeros on the right edge, and for about seventy years the best zero-free region near that edge has been a sliver that gets thinner the higher one goes. The quasi-Riemann hypothesis asks for something weaker but still enormous: a strip of fixed width along the right edge that contains no zeros at all. OpenAI's manuscript claims exactly that for every real part above 7/8, for the zeta function and every Dirichlet L-function, and a companion paper excludes the troublesome Landau-Siegel zeros uniformly.

The reaction among number theorists was close to awe. Stanford's Jared Duker Lichtman called it "the most significant AI-for-math achievement so far, by quite a margin," and Maksym Radziwiłł told Quanta that most consequences of the full Riemann hypothesis already follow from the quasi version. James Maynard called the result groundbreaking and noted that it combines ideas that were not mainstream in an exceptionally clever way. The most revealing detail is where those ideas came from: the proof leans on techniques for cubic Gauss sums that Alexander Dunn and Radziwiłł had developed years earlier, so it introduced no new concepts but exposed a hidden connection that the field had dismissed as too weak. This was also the one manuscript in the release edited by humans before publication, and OpenAI has not named the collaborators. Kontorovich has stressed that the method fundamentally cannot reach 1/2, so the full Riemann hypothesis still needs an idea nobody, human or machine, has yet produced.

Navier-Stokes and Fluid Singularities

The Navier-Stokes equations are Newton's F = ma written for fluids: they track how the velocity and pressure of water or air change in space and time. They are easy to state and brutally hard to analyze, which matters for everything from weather to aircraft drag. The Clay Millennium Prize problem offers four alternatives: prove that smooth solutions exist forever in the unforced case (statements A and B), or show that solutions can break down (statements C and D). OpenAI's September 8 paper claims C and D: starting from a fluid at rest and pushing it with a smooth, compactly supported force, a spinning, contracting, and stretching vortex develops infinite velocity in finite time while the total energy stays finite. The search reportedly used about 10,000 coordinating AI agents for 88 hours, exchanging 2.7 million messages and about 130 billion tokens, and the Lean formalization took another 17 hours, organized on the Prove2Me platform as a directed acyclic graph of proved and open subgoals.

The result arrived wrapped in controversy. About twelve hours before OpenAI's announcement, NYU mathematician Tristan Buckmaster said that his work with Levent Alpöge on blow-up for the related Euler equations may have reached OpenAI and shaped its prompts; OpenAI said its researchers and agents had not seen that work before it became public. Many analysts regard the forced version as the less interesting branch of the problem, because the force can be engineered to feed the singularity, and the unforced statements A and B remain open. The Clay Mathematics Institute's president, Martin Bridson, said its evaluation would be deliberately unhurried and rigorous, and the institute's rules require publication, two years of public availability, and general acceptance by the community; OpenAI has said it will not claim the prize money. Steven Brunton has noted that even a valid blow-up proof does not directly make simulation, turbulence prediction, or engineering design easier, which remain the practical payoffs of fluid mathematics.

Complexity Theory: Unique Games, Log-Space, and 3SUM

The unique games conjecture, proposed by Subhash Khot in 2002, says that a certain kind of constraint puzzle is hard to solve even approximately. If true, it implies that many approximation algorithms already in use, such as the Goemans-Williamson algorithm for Max-Cut, are the best possible. OpenAI's manuscript claims a deterministic reduction from 3SAT to unique games, which would settle the conjecture outright, along with a separate claim that RL = L, meaning randomness gives no extra power to computers restricted to logarithmic memory. The human side of the story was poignant: an MIT team led by Dor Minzer, hearing rumors in September, rushed a closely related result into print, and unique games expert Dana Moshkovitz said OpenAI's paper was so poorly written that she could not read it without AI help.

The 3SUM result came from Anthropic and took a different path to publication. 3SUM asks whether any three of n numbers sum to zero, and the textbook method takes about n2 steps. For two decades, fine-grained complexity theory has rested on the hypothesis that no algorithm can do polynomially better, together with a similar hypothesis that all-pairs shortest paths (APSP) needs about n3 steps; hundreds of other problems were proved "hard" by reduction to these. An internal Anthropic model, asked to check and improve cryptographic constructions, found a way to compute only the needed entries of thin rectangular matrix products, which yields 3SUM in about n1.9992 time and APSP in about n2.9995. Anthropic shared the algorithm under a confidentiality agreement with Josh Alman and Virginia Vassilevska Williams, compensated them for extending and writing it up, and published a Lean formalization. Within two weeks other researchers had simplified the proof and improved the exponents, some using Claude and ChatGPT themselves. As Ankur Moitra observed, the result does more than add algorithms: it reorganizes a field built on the assumption that these problems rise and fall together.

Faster Matrix Multiplication: From 2.371 to 9/4

The October release's matrix multiplication claims are the AI discoveries closest to this page's first focus. A September 24 manuscript claimed ω < 2.258, and an October 2 follow-up claimed ω ≤ 9/4 = 2.25, compared with the August 2026 record of 2.371177; the argument runs through Volker Strassen's theory of the asymptotic spectrum rather than the laser method that produced every record since 1990. An independent researcher rebuilt the Lean formalization and confirmed that it compiles with only Lean's standard axioms, and on October 27 Amir Shpilka posted an elementary, self-contained proof that produces explicit algorithms, while another preprint extended the bound to fields of every characteristic. Shpilka's explicit construction also shows why practitioners should not expect faster GPUs: for exponents within ε of 9/4, the base matrices can need size up to 2(1/ε)O(1). The details and the full history of the exponent are in Matrix Multiplication Algorithm.

Formalization at Scale: Fermat's Last Theorem

Andrew Wiles proved Fermat's Last Theorem in 1995, and in 2024 Kevin Buzzard of Imperial College London began a five-year, grant-funded community project to formalize a modern proof in Lean. In September 2026, Anthropic reported that dozens of Claude agents, coordinated through Prove2Me, had finished an end-to-end formalization in 11 days: about 13 million lines of Lean, more than five times the size of Mathlib, proving 30,300 theorems of which about 29,500 appear in the final proof, using roughly six billion output tokens. The proof rests only on Lean's three standard axioms, and a comparator confirmed that its final statement matches Mathlib's own statement of the theorem. Buzzard compiled it himself and said the achievement was a big step toward automatically formalizing the modern mathematical literature. It is verification, not discovery: the formalization follows the early literature on Wiles's proof and adds no new mathematics, and Kontorovich has described it as "jerry-rigged" code that still needs years of human work to reach library quality. Its significance is that the slowest step in trusting a long proof, checking it line by line, may no longer be the bottleneck.

How Solid Are the Claims?

It helps to think of trust in an AI result as a ladder with five rungs:

  1. A natural-language manuscript exists and claims a theorem.
  2. A Lean formalization of the stated theorem compiles without gaps.
  3. Humans have confirmed that the formal statement really says what the paper claims.
  4. Experts understand the argument well enough to explain, simplify, and extend it.
  5. The community has accepted the result through peer review, talks, and use.

Most autumn 2026 results sit on the second rung, a few (quasi-RH, 3SUM, the 9/4 bound) have climbed to the fourth, and none has yet completed the fifth. The gaps between rungs are real. Quanta reported that some formalizations in the October release prove weaker statements than their papers claim, and an October preprint showed that AI autoformalization can turn an incorrect natural-language proof into a different, correct Lean proof, so a compiling certificate validates the theorem but not necessarily the human-readable argument. There is also survivorship bias: roughly 4,000 problems were attempted and fewer than one in ten produced a released result, and the failures were not published. MIT mathematician Andrew Sutherland has advised treating claims of single-agent "one-shotting" as unverified until the model is released and results can be replicated. Even so, the overall situation is new and important: for the first time, trust at the level of theorems can run ahead of human understanding, and the hard work has shifted to closing that gap. Melanie Wood of the advisory group convened around the release described it as the beginning, not the completion, of the process of human understanding.

How AI Is Reframing Mathematics and Its Institutions

Mathematics has institutions built for a world where proofs are rare and expensive: journals that referee slowly, prizes for first solutions, doctoral training organized around attacking a known open problem, and a culture of openly sharing partial ideas. The autumn 2026 releases put every one of those under strain at once. Ankur Moitra summarized the feeling as being pulled in two directions, thrilled by a tool that acts like a world expert explaining other fields in your own language, yet worried that the playbook for training young researchers had been thrown out overnight. This section looks at how the field is responding.

From Proof Scarcity to Proof Abundance

Terence Tao's framework, laid out in a Caltech lecture on October 9, 2026, contrasts "Math 1.0," an era of proof scarcity in which solving a famous problem first was the highest achievement, with a "Math 2.0" in which proofs are abundant and the scarce goods are understanding, exposition, and good questions. In his view, open problems have served as lighthouses that guide exploration of the mathematical landscape, and reaching them prematurely by brute force can destroy the exploration they were meant to inspire. He wrote that he was horrified to see years of patient research by students and postdocs "casually disrupted or destroyed by such a release." He has also stated plainly that results arriving without explanation, citations, and people willing to answer questions will not be treated as part of the mathematical literature, and he has complained that AI companies do not disclose their failed attempts. Math 2.0, as he describes it, would reward exposition, community building, and opening new directions; would pose "open interpretation problems" about understanding results that are already proved; and would use AI with more imagination than simply pointing agents at famous conjectures.

Journals, Preprints, and Peer Review

Mathematical journals were overloaded before AI, with exhausted referees asked to check ever longer papers, and the community's way of establishing truth is deliberately slow. Kontorovich points to Grigori Perelman's proof of the Poincaré conjecture, which was accepted only after Perelman lectured around the world and experts spent years working through it line by line. A release of 722 manuscripts in one evening does not fit that model, and OpenAI responded by publishing on GitHub with revision and citation protocols and, after criticism, committing on October 15 to better exposition and citation in future releases. Several alternatives are emerging. Moitra has called for collaborative, crowdsourced verification and for papers that can be annotated and improved by the community, while Kontorovich points to Mathlib, where contributions arrive as public GitHub pull requests that maintainers review in the open, much like open-source software. A likely future paper has two parts: a formal certificate that handles correctness, and a human story that explains why the proof works, which shifts what publishing is for.

Credit, Openness, and the Ethics of Release

The release debates turned quickly from mathematics to ethics. The June 2026 Leiden Declaration had already asked for disclosure, verifiability, and human responsibility for attribution. On September 11, after the Navier-Stokes announcement and its priority dispute, more than two dozen Fields Medalists signed "A Severe Misalignment of AI in Mathematics," arguing that using famous open problems as AI benchmarks harms the discipline. An independent Advisory Group on Mathematics and Artificial Intelligence, nine mathematicians hosted at the Institute for Advanced Study, published guidelines on September 29 after more than 600 responses from the community: labs should disclose the model, prompts, a summarized chain of thought, time, cost, and how many comparable problems they tried and failed, and a lab that releases results humans do not yet understand should fund the work of understanding them. OpenAI's October release followed many of the documentation norms but not the guidelines' central warning against a headline dump from a closed model, and the Association for Human Mathematics called it a "demonstration of power" and urged colleagues to stop working with the company. Anthropic's choice to contract and pay human experts to write up the 3SUM result was praised by some as responsible release and criticized by others as outsourcing credibility. These disputes belong to the broader field of AI governance and research ethics, and they are far from settled.

A quieter worry concerns openness itself. Mathematicians have traditionally shared half-formed ideas freely, but Moitra has warned that researchers may become secretive if a single published idea can be fed to an AI and exhausted before its originator can explore it. Kontorovich's hope is the opposite: that the race to be first through the gate fades in importance while the culture of openness survives. Which way the incentives settle will depend on how hiring committees, prize juries, and journals decide to value contributions that are explanatory rather than first.

Careers, Training, and Education

The human cost has been described in stark terms. Buckmaster said whole research programs had been wiped out in a day, Fields Medalist Hugo Duminil-Copin told Le Monde the release had annihilated his field of research, and Scott Aaronson compared mathematicians to hunter-gatherers who woke to find a luxury resort built beside their forest. Others are more hopeful: younger researchers such as Michael Chapman have said they feel more excited than afraid, and Steven Brunton emphasizes that curiosity, solving interesting problems, and helping others remain the real reasons to do mathematics. Practical advice is converging on breadth over narrow specialization, since AI rewards people who can connect fields; on learning to ask good questions, which Brunton and Moitra both argue requires deep mathematical insight; and on what Tao calls controlling one's "AI diet," using tools in ways that build understanding rather than replace it.

Teaching is changing too. Kontorovich now teaches real analysis as a story, starting with calculations on the familiar number line and introducing the formal construction of the real numbers only when a theorem forces the question, an approach that separates the narrative of mathematics from its engineering. Formal proof languages such as Lean, already taught at Imperial College and elsewhere, are becoming a standard part of the toolkit, and Tao has proposed educational materials that integrate new research results directly. In the language of this wiki, the emerging model is a human-in-the-loop or symbiotic partnership, sometimes called "centaur mathematics," in which humans choose directions and interpret meaning while machines search and verify.

A New Lens: Making Mathematics More Transparent

AI has also given mathematics a new tool for transparency, and this may prove to be its most lasting contribution. Formal verification changes whom you must trust: instead of trusting an author, a referee, or 2.7 million agent messages, a reader can trust a small, well-understood checking kernel and focus attention on whether the final theorem statement and its definitions are right. Formal libraries also make hidden context explicit. As Kontorovich notes, a lemma in an ordinary paper silently assumes the previous sixty pages, while a formal statement carries its full specification and cannot be misapplied, which lets results be reused like engineered parts. In this sense the 13-million-line Fermat proof is not meant to be read; it is meant to make every step inspectable by anyone with a computer.

Explanatory transparency is improving even faster. Within three weeks of the October release, human mathematicians, often working with AI, had produced an elementary proof of the 9/4 bound with explicit algorithms, an interactive web page explaining at a high-school level where the number 9/4 comes from, a direct proof of the Landau-Siegel consequence with explicit constants, a simplified proof of the 3SUM result, and an undergraduate-level exposition of the cycle double cover proof. Brunton's metaphor captures what else is happening: even if AI only recombines known ideas, it lifts the fog over the landscape, showing which problems were within reach of existing tools all along and revealing connections, like the Dunn-Radziwiłł techniques behind quasi-RH, that specialists had overlooked. Tools for literature-based discovery and semantic search extend that effect to the whole published record.

Transparency has limits, and they matter. A proof can be fully formal and still incomprehensible, and Moitra has asked what it would mean to "unformalize" such proofs, recovering the special cases and intuitions humans need. Mechanistic studies of how models compute, described in Mathematical Reasoning and the broader field of Explainable AI, may eventually explain why a model chose one chain of moves over another. Until then, Kontorovich's summary stands as the right standard: "Math is not the fact, it's the why."

Did AI Expand Mathematics or Just Connect the Dots?

The question sounds simple: did AI genuinely expand our understanding of mathematics, or did it connect existing dots by sophisticated pattern matching? The honest answer depends on what "expand" means, because mathematics grows in at least five ways: new true statements, new techniques, new concepts and definitions, new questions, and deeper human understanding. The autumn 2026 results look very different depending on which of these one measures, and serious mathematicians disagree. The two sides are set out below, followed by an attempt to reconcile them.

The Case for Recombination

The skeptical view holds that today's systems mostly recombine what humans already invented. Kontorovich, who called the quasi-RH proof Fields Medal caliber had a human found it, also judged that it contains no genuinely new idea: it is a long, beautiful chain of moves that analytic number theorists already knew, built on recent human breakthroughs on cubic Gauss sums. Critics describe this as staying inside the "convex hull" of human knowledge, like air moving around inside a balloon without pushing its walls outward, and they note that AI has not yet handed mathematicians a new definition or concept that reorganizes a subject. The model has read essentially every paper, which gives it unmatched breadth, so many "discoveries" may be retrieval and assembly at superhuman scale, as the early Erdős episodes showed when apparent solutions turned out to be forgotten literature.

There is also evidence that AI succeeds differently from humans. Moitra recalls research showing that image classifiers rely on statistical features humans cannot see, the same features exploited by adversarial examples, and he observes that OpenAI's solutions do not look like human solutions. Studies such as GSM-Symbolic found that models stumble when irrelevant details change, and roughly nine in ten problems attempted for the October release produced no published result. On this view, AI is an extraordinarily powerful search engine over known techniques, guided by tree search and reinforcement learning, rather than a source of mathematical ideas.

The Case for Genuine Expansion

The opposing view starts by questioning the balloon metaphor. Kontorovich prefers a framing he credits to Tim Gowers: AI explores the semigroup generated by known moves, like laying tiles one after another, and such combinations can reach regions far beyond anything humans explored. The history of mathematics includes famous problems that fell to short, elementary arguments that could have been found decades earlier; reaching them was still a real discovery. Radziwiłł concluded that number theorists had simply been wrong to think their tools were too weak for quasi-RH, which means the AI revealed something true about the power of existing mathematics that no human had seen. The size of the convex hull, Moitra notes, was itself unknown.

Some results also changed what mathematicians will study next, which is a strong test of expansion. The collapse of the 3SUM and APSP hypotheses reorganizes fine-grained complexity theory, the claimed unique games proof would settle the limits of efficient approximation, and a drop in the matrix multiplication exponent from about 2.371 to 2.25 is the largest in decades. Brunton points out that systems already perform automatic discovery of compact physical laws from data, using methods such as symbolic regression, and he expects AI to start suggesting which questions to ask. Even pure "dot connecting" at this scale makes the landscape visible in a way it never was, and the AlphaGo lineage (AlphaGo and its successors) showed that search over known rules can still produce moves experts later call creative.

Reconciling the Views

Kind of expansion Evidence it happened Evidence it has not happened yet
New true statements Hundreds of claimed theorems, many Lean-checked, including quasi-RH, ω ≤ 9/4, and the refutation of the 3SUM hypothesis Most attempted problems failed, and failures were not published
New techniques Unexpected chains of known moves, and thin-product tricks that others immediately reused for APSP and related problems Experts trace most proofs to existing tools, such as the Dunn and Radziwiłł methods behind quasi-RH
New concepts and definitions None widely recognized so far Kontorovich notes AI has not yet offered a genuinely new definition
New questions and directions Whole subfields reoriented overnight, such as fine-grained complexity after 3SUM fell Humans still chose the problems; AI has not yet told mathematicians what to ask
Human understanding Fast human expositions, such as the elementary explicit-algorithm proof of the 9/4 bound Many manuscripts remain unread; Tao describes the bottleneck moving from generation to verification to digestion

Put together, the evidence supports a two-part answer. AI has unmistakably expanded the set of known mathematical truths and has shown that the region reachable by existing techniques is far larger than anyone believed. It has not yet clearly expanded mathematics in the conceptual sense of new definitions, new theories, or new kinds of questions, and its contributions to human understanding so far come mostly from humans digesting what it produced. Moitra's "doorman fallacy" is a useful warning here: just as a doorman does far more than open doors, a mathematician does far more than prove theorems, and automating the most visible task does not automate the job. Kontorovich has posed a thought experiment he calls the "solved axiom," supposing that any problem a human and AI could solve together, AI could solve alone; he does not believe it, but argues that even in that world pure mathematics would keep its value, because that value lives in human minds rather than in data centers.

There is also a hard mathematical limit on the most extreme fears. Gödel's incompleteness theorems guarantee that any consistent formal system strong enough for arithmetic contains true statements it cannot prove, so no machine working inside Lean's axioms will ever finish mathematics, and choosing which questions and axioms are worth having remains a human judgment. The most likely near future is the one Brunton calls hopeful and Tao calls Math 2.0: machines expand the map of known results at astonishing speed, while humans decide where to look, what the results mean, and how to explain them. Whether AI eventually supplies the missing ingredient, a genuinely new idea of the kind that the full Riemann hypothesis seems to require, is the open question that will define the next decade.

Mathematics - Getting Started

If you are starting from scratch, a practical order is: (1) linear algebra, through 3Blue1Brown's "Essence of linear algebra" and Gilbert Strang's MIT course; (2) single and multivariable calculus, especially derivatives, the chain rule, and gradients; (3) probability and statistics, through Khan Academy and StatQuest; (4) optimization, starting with gradient descent; (5) the machine-learning synthesis in Mathematics for Machine Learning or Michael Nielsen's free online book; and (6) Transformers, through 3Blue1Brown's deep learning chapters 5 to 7. You do not need to master each step before starting the next; most people learn the math faster once they see where it is used. The video tables below follow roughly the same order, starting with why mathematics matters at all.

Load video
YouTube
Math is the hidden secret to understanding the world | Roger Antonsen
Unlock the mysteries and inner workings of the world through one of the most imaginative art forms ever (mathematics) with Roger Antonsen, as he explains how a slight change in perspective can reveal patterns, numbers and formulas as the gateways to empathy and understanding. I am a logician, mathematician, computer scientist, author, public speaker, science communicator, and artist. You can find me at the University of Oslo, where I teach Logical Methods as an Associate Professor at the Department of Informatics in the research group Analytical Solutions and Reasoning (ASR), otherwise at UC Berkeley, California and ICERM at Brown University, where I am a Visiting Scholar. I am also engaged in various forms of science communication and outreach, which you may read about below. My academic interests are logical calculi, proof theory, mathematical logic, complexity theory, automata, combinatorics, philosophy of mathematics, visualizations, and mathematical art, but I am interested in most topics related to mathematics, computer science, art, and philosophy.

Load video
YouTube
Mathematics is the sense you never knew you had | Eddie Woo | TEDxSydney
In this illuminating talk, high school mathematics teacher and YouTube star Eddie Woo shares his passion for mathematics, declaring that "mathematics is a sense, just like sight and touch" and one we can all embrace. Using surprising examples of geometry, he encourages everyone to seek out the patterns around us, for "a whole new way to see the world". A public high school teacher for more than 10 years, Eddie Woo gained international attention when he posted videos of his classroom lessons online, to assist an ill student. His YouTube channel, WooTube, has more than 200,000 subscribers and over 13 million views. Eddie believe that mathematics can be embraced and even enjoyed by absolutely everybody. He was named Australia's Local Hero and was a Top 10 Finalist in the Global Teacher Prize for his love of teaching mathematics. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at https://www.ted.com/tedx

Load video
YouTube
How you can be good at math, and other surprising facts about learning | Jo Boaler | TEDxStanford
You have probably heard people say they are just bad at math, or perhaps you yourself feel like you are not “a math person.” Not so, says Stanford mathematics education professor Jo Boaler, who shares the brain research showing that with the right teaching and messages, we can all be good at math. Not only that, our brains operate differently when we believe in ourselves. Boaler gives hope to the the mathematically fearful or challenged, shows a pathway to success, and brings into question the very basics of how our teachers approach what should be a rewarding experience for all children and adults. Jo Boaler is a professor of mathematics education at Stanford and the co-founder of YouCubed, which provides resources and ideas to inspire and excite students about mathematics. She is also the author of the first massive open online course on mathematics teaching and learning. Her book Experiencing School Mathematics won the Outstanding Book of the Year award for education in Britain. A recipient of a National Science Foundation "early career award"' she was recently named by BBC as one of the eight educators changing the face of education. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at http://ted.com/tedx

Load video
YouTube
Love and Math an interview with Edward Frenkel
UC Professor of mathematics Edward Frenkel describes the relationship of Love and Mathematics, calls for a more modern way of teaching math in schools, and talks of the principles and people that have advanced our understanding of Math as a window onto reality. Edward Frenkel is a professor of mathematics at the University of California, Berkeley, which he joined in 1997 after being on the faculty at Harvard University. He is a member of the American Academy of Arts and Sciences, a Fellow of the American Mathematical Society, and the winner of the Hermann Weyl Prize in mathematical physics. Frenkel has authored three books and over eighty scholarly articles in academic journals, and he has lectured on his work around the world. His YouTube videos have garnered over 4 million views combined. Frenkel’s latest book Love and Math was a New York Times bestseller and has been named one of the Best Books of the year by both Amazon and iBooks. It is being translated into 16 languages. Frenkel has also co-produced, co-directed and played the lead in the film Rites of Love and Math (2010).

Mathematics Ontology

The table below maps the main branches of mathematics to the jobs they do inside AI. Most real systems use all of them at once. The last row, logic and discrete mathematics, is also the branch where AI has made its most visible contributions back to mathematics.

Area What it does in AI Where you meet it
Linear algebra Stores and transforms data as vectors, matrices, and tensors Embeddings, layers, attention, PCA, recommendation systems
Calculus Measures how outputs change when parameters change Gradients, backpropagation, optimizers, normalizing flows
Probability Represents uncertainty and turns scores into predictions Softmax, cross-entropy, Bayes rule, sampling, diffusion models
Statistics Separates signal from noise in data and in results Train and test splits, confidence intervals, p-values, benchmark evals
Optimization Finds parameters that minimize a loss Gradient descent, Adam, learning-rate schedules, reinforcement learning
Information theory Counts bits and measures surprise Cross-entropy, perplexity, compression, quantization
Fourier and signal analysis Moves between time or space and frequency Convolutions, positional encodings, audio, neural operators
Logic and discrete mathematics Structures exact reasoning, graphs, and proof Search, knowledge graphs, formal verification in Lean

Load video
YouTube
The Map of Mathematics
The entire field of mathematics summarised in a single map! This shows how pure mathematics and applied mathematics relate to each other and all of the sub-topics they are made from.

Load video
YouTube
Mind Map of Maths
LarryLemonMaths

Mathematics for Machine Learning | M. Deisenroth, A Faisal, and C. Ong .. Companion webpage ...


The Roadmap of Mathematics for Deep Learning | Tivadar Danka ...Understanding the inner workings of neural networks from the ground-up

Scalar, Vector, Matrix & Tensor

These four objects differ mainly in how many indices you need to point at a single number. A scalar needs none, a vector needs one, a matrix needs two, and a tensor can need three or more. In deep learning frameworks they are all "tensors" of different rank: a loss value has shape (), an embedding has shape (768,), a weight matrix has shape (768, 3072), and a batch of token embeddings might have shape (32, 2048, 768).

Load video
YouTube
How Data travels in Deep Neural Networks | Scalar vs Vector vs Matrix vs Tensor
This video titled "How Data travels in Deep Neural Networks | Scalar vs Vector vs Matrix vs Tensor" explains the role of Tensors in TensorFlow, What exactly are tensors as well as Characteristics of tensors. How Data travels in Deep Neural Networks? A comparison of Scalar vs Vector vs Matrix vs Tensor. SUPPORT ME on Patreon: http://www.patreon.com/theaiuniversity

Load video
YouTube
Why Linear Algebra ? Scalars, Vectors, Tensors
NPTEL-NOC IITM Lecture - 03

Load video
YouTube
Transformation properties of scalars, vectors and tensors
This video is on the basics of scalars, vectors and tensors and their transformation properties.

Load video
YouTube
Scalar, Vector, Matrix, Tensor, Matrix Transpose
Deep Shallownet Topics to cover based on: Deep Learning An MIT Press book Ian Goodfellow and Yoshua Bengio and Aaron Courville http://www.deeplearningbook.org/


Scalars

A scalar is a single number. For example, an individual weight in a neural network is denoted by just one number. Other everyday scalars in AI include the learning rate, the value of the loss, a sampling temperature, and a single probability. In code, a scalar is a tensor with zero dimensions.


Vector

Vectors are an array of numbers. The numbers are arranged in order and we can identify each individual number by its index in that ordering. We can think of vectors as identifying points in space, with each element giving the coordinate along a different axis. In simple terms, a vector is an arrow representing a quantity that has both magnitude and direction wherein the length of the arrow represents the magnitude and the orientation tells you the direction. For example wind, which has a direction and magnitude.

In AI, vectors carry meaning. A word, image, or document becomes an embedding vector, and things with similar meaning end up pointing in similar directions. The dot product a·b = Σ aibi measures how aligned two vectors are, and dividing by both lengths gives cosine similarity, the workhorse of semantic search and retrieval. The length of a vector is measured with a norm: the L2 norm ‖x‖2 = √(Σ xi2) is ordinary distance, and the L1 norm Σ |xi| is used to encourage sparsity. Hyperdimensional Computing (HDC) pushes the idea further, computing with vectors that have thousands of dimensions.

Load video
YouTube

Load video
YouTube
What is a vector? - David Huynh
Physicists, air traffic controllers, and video game creators all have at least one thing in common: vectors. But what exactly are they, and why do they matter? David Huynh explains how vectors are a prime example of the elegance, beauty, and fundamental usefulness of mathematics. Lesson by David Huynh, animation by Anton Trofimov.

Matrices

A matrix is a 2D-array of numbers, so each element is identified by two indices instead of just one. If a real-valued matrix A has a height of m and a width of n, then we say that A ∈ ℝm×n. We identify the elements of the matrix as Ai,j, where i is the row (from 1 to m) and j is the column (from 1 to n).

A matrix is best understood as a function: multiplying a vector by A rotates, stretches, projects, or shears it. Composing two such functions is the same as multiplying their matrices, which is why deep networks are built from chains of matrix products. A few matrix ideas show up everywhere in AI:


Applications of Matrices

Load video
YouTube
The Applications of Matrices | What I wish my teachers told me way earlier
Zach Star This video goes over just a few applications of matrices that may give you some insight into how they can be used in the real world. Linear algebra was never explained well to me in school and I had very little motivation to learn matrices at the time so hopefully this helps you if you're in a similar situation. Also note there are so many applications there's just no way to fit them into one video but here you'll find some of my favorite instances of when they come up. Support the Channel: http://www.patreon.com/zachstar Google PageRank Algorithm: http://youtu.be/qxEkY8OScYY Mathematics Used to Solve Crime: http://youtu.be/-cXBgHgX5UE

Load video
YouTube
Dear linear algebra students, This is what matrices (and matrix manipulation) really look like
Sign up with brilliant and get 20% off your annual subscription: http://brilliant.org/ZachStar/ Support the Channel: http://www.patreon.com/zachstar 3D Software Used Runiter (for the actual graphs): http://www.runiter.com/ Geogebra (used for vectors, & is free): http://www.geogebra.org/3d?lang=en Animations: Brainup Studios ( http://brainup.in/ )

Matrix Multiplication Algorithm

To multiply an m×k matrix A by a k×n matrix B, each entry of the result is a dot product: Ci,j = Σk Ai,kBk,j. The schoolbook method therefore needs n3 multiplications for two n×n matrices. In 1969 Volker Strassen found that two 2×2 matrices can be multiplied with 7 multiplications instead of 8, at the cost of extra additions. Applied recursively to blocks, that gives an algorithm with about n2.807 operations (log27 ≈ 2.807). The smallest exponent achievable, called ω, is one of the central open problems in theoretical computer science. Everyone agrees that ω ≥ 2, since you must at least read all n2 entries, and many researchers suspect ω = 2.

Year Result Upper bound on ω
Before 1969 Schoolbook method 3
1969 Strassen 2.807
1990 Coppersmith and Winograd 2.376
2023 Duan, Wu, and Zhou 2.371866
2024 Vassilevska Williams, Xu, Xu, and Zhou 2.371552
2025 Alman, Duan, Vassilevska Williams, Xu, Xu, and Zhou 2.371339
2026 (August) Dupont et al., with AlphaEvolve (preprint) 2.371177
2026 (October) OpenAI internal model (claimed; Lean-formalized; human exposition by Shpilka) 2.25 (9/4), over the complex numbers

The asymptotic algorithms behind the bounds below 2.8 are "galactic": their hidden constants are so large that they only win for matrices far bigger than any computer can hold. They matter for theory, and they show that the limits of fast computation are still unknown. Shpilka's explicit version of the 9/4 algorithm makes the point vividly, since getting within ε of the exponent can require base matrices of size up to 2(1/ε)O(1). Practical speedups come from Strassen-like schemes for small blocks, from hardware, and from doing less multiplication altogether.

In deep learning, matrix multiplication is essential for combining data and weights in neural networks. MatMul operations are vital for transforming input data through the layers of a neural network, facilitating predictions during training and inference. GPUs, with their highly parallel architecture, are designed to execute numerous MatMul operations simultaneously. This parallelism enables GPUs to manage the large-scale computations required in deep learning far more efficiently than traditional CPUs, making them crucial for training and deploying complex neural network models. Modern accelerators include dedicated matrix units (tensor cores on GPUs, matrix units on TPUs) that perform small block multiplications in a single instruction, often in reduced precision.

In traditional Transformer models, self-attention uses MatMul operations to compute relationships between all pairs of tokens to capture dependencies and contextual information, and the feed-forward layers that hold most of the parameters are dense MatMuls too. Matrix multiplications are therefore the most computationally intensive operations in large language models built on the Transformer architecture. As LLMs grow to hundreds of billions of parameters, the cost of MatMul increases dramatically, leading to higher memory usage and longer latency during both training and inference, and MatMul has become a significant bottleneck that necessitates extremely large GPU clusters. The physics of that cost is explained in The Physics.

Load video
YouTube
Multiplying Matrices
This precalculus video tutorial provides a basic introduction into multiplying matrices. It explains how to tell if you can multiply two matrices together and how to determine the order of the new matrix. The order of the new matrix is based on the rows of the first matrix and the number of columns in the second matrix. To multiply two matrices together, multiply the rows of the first matrix by the columns of the second matrix and find the sum of the products. This video contains plenty of examples and practice problems on matrix multiplication.

Load video
YouTube
How to organize, add and multiply matrices - Bill Shillito
When you're working on a problem with lots of numbers, as in economics, cryptography or 3D graphics, it helps to organize those numbers into a grid, or matrix. Bill Shillito shows us how to work with matrices, with tips for adding, subtracting and multiplying (but not dividing!). Lesson by Bill Shillito, animation by The Leading Sheep Studios.


Alternative Matrix Algorithms

Researchers are investigating several alternatives to dense, full-precision matrix multiplication in Large Language Models (LLMs). They fall into two families: computing the same product with fewer or cheaper operations, and changing the model so that it needs less multiplication in the first place. Some notable approaches include:

  • Tensor Decomposition Search: Google DeepMind's AlphaTensor (2022) and AlphaEvolve (2025) search for new ways to write matrix multiplication with fewer scalar multiplications by finding low-rank decompositions of the matrix multiplication tensor. Earlier versions of this page called this "tensor reduction"; the standard term is tensor decomposition. See below.
  • Matrix Addition Algorithm: The MatMul-free architecture eliminates dense matrix multiplications from language models while maintaining strong performance at the scales tested. It combines the MatMul-free Linear Gated Recurrent Unit (MLGRU) token mixer and the Gated Linear Unit (GLU) channel mixer with ternary weights.
  • Ternary and Low-Bit Weights: Microsoft's BitNet b1.58 trains models natively with weights in {-1, 0, +1}, turning most multiplications into additions and subtractions. Its 2-billion-parameter, 4-trillion-token release (BitNet b1.58 2B4T, April 2025) fits its weights in about 0.4 GB and performs comparably to similarly sized full-precision open models on standard benchmarks. Microsoft's technical report estimates decoding energy several times to more than ten times lower than comparable models, though these are modeled estimates, and Microsoft does not recommend commercial deployment without further testing.
  • Sparse Attention Mechanisms: Researchers are investigating methods to reduce the computational cost of attention mechanisms by using sparse matrices or attention pruning techniques. These approaches aim to focus computation only on relevant parts of the input sequence. Examples include sliding-window attention, which attends only to recent tokens, and learned sparse patterns such as DeepSeek's Native Sparse Attention (2025). A related but different idea, FlashAttention, computes exact attention while reorganizing the work to cut memory traffic, which is often the real bottleneck.
  • Low-Rank Factorization: Techniques such as low-rank matrix factorization are being explored to approximate the original weight matrices in LLMs. By reducing the rank of these matrices, the computational complexity of matrix operations can be significantly decreased while still preserving model performance to some extent. LoRA fine-tuning adds small low-rank updates instead of changing full matrices, and multi-head latent attention compresses the attention cache through a low-rank projection.
  • Quantization and Compression: Quantization methods aim to reduce the precision of weight matrices in LLMs, thus reducing the memory and computational requirements during inference. Compression techniques, such as matrix decomposition or tensor decomposition, are also investigated to reduce the overall size of the model without sacrificing performance. Post-training methods such as GPTQ and AWQ compress existing models to 4 bits. Since 2025, quantization has moved into training too: NVIDIA's NVFP4 recipe trained a 12-billion-parameter model on 10 trillion tokens in 4-bit floating point with loss and downstream accuracy comparable to an FP8 baseline, while the competing MXFP4 format needed about 36 percent more tokens to reach the same loss.
  • Attention Compression: Some researchers are focusing on compressing attention mechanisms in LLMs by employing techniques like attention factorization, attention sparsity, or hierarchical attention mechanisms. These approaches aim to reduce the computational overhead associated with attention mechanisms while maintaining or improving model performance. Grouped-query attention, which shares key and value matrices across heads, and quantized attention caches are now standard in production models.
  • Conditional Computation: Mixture-of-Experts (MoE) models contain many expert weight matrices but route each token through only a few, so the FLOPs per token are a fraction of the total parameter count.
  • Analog and In-Memory Computing: Instead of moving weights to an arithmetic unit, store them in a memory array and let Ohm's and Kirchhoff's laws compute the matrix-vector product where the data sits. See Analog Computers.
Matrix Tensor-decomposition Algorithm (AlphaTensor and AlphaEvolve)

Matrix multiplication can be written as a fixed three-dimensional array of 0s and 1s, the matrix multiplication tensor. For n×n matrices it has n2×n2×n2 entries. If you can write that tensor as a sum of R rank-one pieces (each the outer product of three vectors), you get an algorithm that uses exactly R scalar multiplications. The schoolbook method corresponds to R = n3; Strassen's 2×2 method is a decomposition with R = 7, and 7 is known to be the minimum for 2×2. Any such decomposition can be applied recursively to blocks, so a better small decomposition improves the exponent for all sizes.

Tensor decomposition achieves this optimization by employing advanced mathematical techniques to simplify and expedite the matrix multiplication process. It leverages tensor algebra, which extends traditional matrix algebra to higher dimensions, enabling more complex representations of data. By exploiting the structure and properties of tensors, these algorithms identify patterns that let fewer computations do the same work.

The savings come from algebra, not from skipping data. Each of the R multiplications multiplies a carefully chosen sum of entries of A by a carefully chosen sum of entries of B; each product then contributes to several output entries, and unwanted terms cancel. Finding decompositions with fewer rank-one terms is a hard search problem, which is why machine learning helps. This selective approach reduces the number of multiplications required without compromising exactness, although extra additions and reduced numerical stability mean the hardware benefit depends on matrix size, number format, and chip design.

AlphaTensor (2022) turned the search into a single-player game and trained an AlphaZero-style agent to play it. It improved the best known rank for many matrix sizes, found a 47-multiplication method for 4×4 matrices in arithmetic modulo 2 (beating two levels of Strassen, which need 49), and discovered algorithms tuned to specific GPUs and TPUs that ran faster in practice on large matrices. AlphaEvolve (2025) is a general-purpose coding agent rather than a specialized solver: Gemini evolves the code of a search procedure, and an evaluator checks each candidate decomposition. It improved the state of the art on 14 matrix multiplication targets and found the first rank-48 decomposition for 4×4 matrices over the complex numbers, improving on Strassen's approach in that setting for the first time since 1969. Algorithms with fewer than 49 multiplications existed before, but they relied on commutativity and cannot be applied recursively to blocks. AlphaEvolve also sped up a matrix-multiplication kernel used to train Gemini by 23 percent, cutting overall training time by about 1 percent. In August 2026, an AlphaEvolve-refined optimizer helped lower the bound on ω itself, and in October an OpenAI model claimed a much larger improvement, both described in Matrix Multiplication Algorithm.

Load video
YouTube

Matrix Addition Algorithm

Researchers at the University of California, Santa Cruz, Soochow University and University of California, Davis have developed a novel architecture that eliminates dense matrix multiplications from language models while maintaining strong performance at the scales they tested, by combining the MatMul-free Gated Recurrent Unit (MLGRU) token mixer and the Gated Linear Unit (GLU) channel mixer with ternary weights. Replacing matrix multiplications (MatMul) with simpler operations can save a lot of memory and computation. However, past attempts to replace MatMul have had mixed results: while they reduced memory usage, they often slowed down operations because they didn't perform well on GPUs. One approach involves replacing the traditional 16-bit floating point weights used in Transformers with ternary weights, which can be -1, 0, or +1. Each ternary weight carries log23 ≈ 1.58 bits of information (an earlier version of this page said "3-bit," which was incorrect). They also replace MatMul with additive operations, achieving similar results with much less computational cost. The models use "BitLinear layers" that utilize these ternary weights, together with 8-bit activations.

By restricting the weights to {-1, 0, +1} and using additional quantization techniques, MatMul operations are replaced with simpler addition and negation operations. They also make significant changes to the language model architecture. Transformer blocks have two main components: a token mixer and a channel mixer. The token mixer integrates information across different tokens in a sequence, using a MLGRU. The GRU is a recurrent sequence modeling technique that was popular before Transformers. The MLGRU processes the sequence of tokens by updating hidden states through simple ternary operations and element-wise gating, eliminating the need for expensive matrix multiplications and for the quadratic attention grid.

The channel mixer integrates information across different feature channels within a single token's representation. Researchers implemented their channel mixer using a GLU, which is also used in models like LLaMA-2 and Mistral. They modified the GLU to work with ternary weights instead of MatMul operations, reducing computational complexity and memory usage while maintaining effective feature integration.

What the experiments showed. Models from 370 million to 2.7 billion parameters were competitive with full-precision Transformers trained the same way, and the gap narrowed as models grew. An optimized GPU implementation cut training memory by up to 61 percent and inference memory by more than a factor of ten. A custom FPGA accelerator ran billion-parameter-scale models at about 13 watts, and later versions of the paper mapped the model onto Intel's Loihi 2 neuromorphic hardware, reporting large gains in energy efficiency.

What it does not yet establish. "MatMul-free" is not "multiplication-free": element-wise products in the gates remain. Results stop at a few billion parameters, and behavior at frontier scale is a projection from scaling curves. Mainstream GPU libraries still lack native ternary kernels, so the biggest gains need custom hardware. The related BitNet work described above shows that ternary weights themselves scale well at least to 2 billion parameters trained on 4 trillion tokens.

Load video
YouTube

Tensors

In mathematics, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space. Objects that tensors may map between include vectors and scalars, and, recursively, even other tensors. Tensors can take several different forms: for example, scalars and vectors (which are the simplest tensors), dual vectors, multi-linear maps between vector spaces, and even some operations such as the dot product. Tensors are defined independent of any basis, although they are often referred to by their components in a basis related to a particular coordinate system. Wikipedia

In machine learning frameworks such as TensorFlow and PyTorch, "tensor" usually means something simpler: an n-dimensional array with a shape and a data type. The physicist's tensor also carries rules for how its components change under a change of coordinates (the covariant and contravariant behavior explained in the videos below), while the programmer's tensor is just a container. Both meanings meet in the matrix multiplication tensor described above, a genuine three-way multilinear object whose decompositions are literal algorithms.

Load video
YouTube
What's a Tensor?
Dan Fleisch briefly explains some vector and tensor concepts from A Student's Guide to Vectors and Tensors

Load video
YouTube
Tensors Explained Intuitively: Covariant, Contravariant, Rank
Tensors of rank 1, 2, and 3 visualized with covariant and contravariant components. My Patreon page is at http://www.patreon.com/EugeneK

Load video
YouTube

3blue1brown

Load video
YouTube
But what is a Neural Network? | Deep learning, chapter 1
Home page: http://www.3blue1brown.com/ Brought to you by you: http://3b1b.co/nn1-thanks Additional funding provided by Amplify Partners Full playlist: http://3b1b.co/neural-networks Typo correction: At 14 minutes 45 seconds, the last index on the bias vector is n, when it's supposed to in fact be a k. Thanks for the sharp eyes that caught that! For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: http://goo.gl/Zmczdy There are two neat things about this book. First, it's available for free, so consider joining me in making a donation Nielsen's way if you get something out of it. And second, it's centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning! http://github.com/mnielsen/neural-networks-and-deep-learning I also highly recommend Chris Olah's blog: http://colah.github.io/ For more videos, Welch Labs also has some great series on machine learning: http://youtu.be/i8D90DkCLhI http://youtu.be/bxe2T-V8XRs For those of you looking to go even deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville. Also, the publication Distill is just utterly beautiful: http://distill.pub/ Lion photo by Kevin Pluck

Load video
YouTube
Gradient descent, how neural networks learn | Deep learning, chapter 2
Home page: http://www.3blue1brown.com/ Brought to you by you: http://3b1b.co/nn2-thanks And by Amplify Partners. For any early-stage ML startup founders, Amplify Partners would love to hear from you via 3blue1brown@amplifypartners.com To learn more, I highly recommend the book by Michael Nielsen http://neuralnetworksanddeeplearning.com/ The book walks through the code behind the example in these videos, which you can find here: http://github.com/mnielsen/neural-networks-and-deep-learning MNIST database: http://yann.lecun.com/exdb/mnist/ Also check out Chris Olah's blog: http://colah.github.io/ His post on Neural networks and topology is particular beautiful, but honestly all of the stuff there is great. And if you like that, you'll love the publications at distill: http://distill.pub/

Load video
YouTube
What is backpropagation really doing? | Deep learning, chapter 3
What's actually happening to a neural network as it learns? Next video: http://youtu.be/tIeHLnjs5U8 Brought to you by you: http://3b1b.co/nn3-thanks And by CrowdFlower: http://3b1b.co/crowdflower Home page: http://www.3blue1brown.com/ The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.

Load video
YouTube
Backpropagation calculus | Deep learning, chapter 4
Brought to you by you: http://3b1b.co/nn3-thanks This one is a bit more symbol heavy, and that's actually the point. The goal here is to represent in somewhat more formal terms the intuition for how backpropagation works in part 3 of the series, hopefully providing some connection between that video and other texts/code that you come across later. 3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that): http://3b1b.co/subscribe If you are new to this channel and want to see more, a good place to start is this playlist: http://3b1b.co/recommended

Determinants

The determinant of a matrix describes how the volume of an object scales under the corresponding linear transformation. If the transformation changes orientations, the sign of the determinant is negative. For a 2×2 matrix with rows (a, b) and (c, d), the determinant is ad - bc. Two facts make it useful: det(AB) = det(A)·det(B), and a matrix can be inverted exactly when its determinant is not zero, because a zero determinant means the transformation squashes space into a lower dimension and information is lost.

For a nonlinear map, the Jacobian matrix collects all first partial derivatives, and its determinant measures how the map scales tiny volumes at each point. AI uses this in normalizing flows, generative models whose likelihood comes from the change-of-variables formula: log px(x) = log pz(z) + log |det ∂z/∂x|. The Jacobian determinant is also the subject of the Jacobian conjecture, which asked whether a polynomial map with a constant nonzero Jacobian determinant must always be invertible. An AI-assisted counterexample in 2026 showed that the answer is no in three or more dimensions; see Recent Developments.

Load video
YouTube
The determinant | Essence of linear algebra, chapter 6
Home page: http://www.3blue1brown.com/ The determinant of a linear transformation measures how much areas/volumes change during the transformation. Full series: http://3b1b.co/eola Future series like this are funded by the community, through Patreon, where supporters get early access as the series is being produced. http://3b1b.co/support 3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted about new videos, subscribe, and click the bell to receive notifications (if you're into that). If you are new to this channel and want to see more, a good place to start is this playlist: http://goo.gl/WmnCQZ Various social media stuffs: Website: http://www.3blue1brown.com

Explained

Load video
YouTube
The Mathematics Of Intelligence | Geoff Goodhill | TEDxUQ
Professor Goodhill's research aims to discover the computational rules underlying brain development and function. He originally trained in the UK in maths, physics and artificial intelligence, and then spent 10 years researching in the USA, including 8 as a professor of neuroscience at Georgetown University. He moved to the University of Queensland in 2005, where he holds a joint appointment between the Queensland Brain Institute and School of Mathematics and Physics. His lab uses experimental, mathematical and computational techniques to understand the brain as a computational device. Professor Goodhill did a Joint Honours BSc in Mathematics and Physics at Bristol University (UK), followed by an MSc in Artificial Intelligence at Edinburgh University and a PhD in Cognitive Science at Sussex University. Following a postdoc at Edinburgh University he moved to the USA in 1994, where he did further postdoctoral study in Computational Neuroscience at Baylor College of Medicine and the Salk Institute. Professor Goodhill formed his own lab at Georgetown University in 1996, where he was awarded tenure in the Department of Neuroscience in 2001. In 2005 he moved to a joint appointment between the Queensland Brain Institute and the School of Mathematical and Physical Sciences at the University of Queensland. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at http://ted.com/tedx

Load video
YouTube
Connections between physics and deep learning
MITCBMM Max Tegmark - MIT

Siraj Raval

These short, energetic videos introduce the math behind machine learning, running from linear algebra and calculus through probability and statistics, so they work best as quick previews before a full course. A note for readers: in 2019 Raval publicly acknowledged plagiarism in a research paper and faced criticism over a paid course, so it is worth pairing these videos with the primary sources and courses linked elsewhere on this page.

Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube

Gilbert Strang (MIT) - Linear Algebra

Strang's lectures are probably the most watched linear algebra course in history. His emphasis on the four fundamental subspaces, factorizations such as A = LU, A = QR, and A = UΣVT, and "the column picture" of matrix multiplication maps directly onto how neural networks are built and analyzed. His later book Linear Algebra and Learning from Data (2019) and the 18.065 course were written specifically for machine learning. Strang retired from MIT teaching in 2023, and his recorded courses remain freely available.

Load video
YouTube
Course Introduction | MIT 18.06SC Linear Algebra
Instructor: Gilbert Strang View the complete course: http://ocw.mit.edu/18-06SCF11 Professor Gil Strang describes the key concepts of undergraduate course Linear Algebra, who should take it, and how it is taught. He provides examples of applications of linear algebra and how it is useful in physics, economics and social sciences, natural sciences, and engineering. License: Creative Commons BY-NC-SA More information at http://ocw.mit.edu/terms More courses at http://ocw.mit.edu

Load video
YouTube
A conversation with Gilbert Strang
Gilbert Strang was an undergraduate at MIT and a Rhodes Scholar at Balliol College, Oxford. His Ph.D. was from UCLA and since then he has taught at MIT. He has been a Sloan Fellow and a Fairchild Scholar and is a Fellow of the American Academy of Arts and Sciences. He is a Professor of Mathematics at MIT, an Honorary Fellow of Balliol College, and a member of the National Academy of Sciences. Professor Strang has published eleven books.

Fourier Transform (FT), Fourier Series, and Fourier Analysis

Joseph Fourier showed that representing a function as a sum of trigonometric functions greatly simplifies the study of heat transfer. Joseph was a French mathematician and physicist born in Auxerre and best known for initiating the investigation of Fourier series, which eventually developed into Fourier analysis and harmonic analysis, and their applications to problems of heat transfer and vibrations. The Fourier transform and Fourier's law of conduction are also named in his honor. Fourier is also generally credited with the discovery of the greenhouse effect.

  • Fourier Transform (FT) decomposes a function of time (a signal) into its constituent frequencies. This is similar to the way a musical chord can be expressed in terms of the volumes and frequencies of its constituent notes. Fourier Transform | Wikipedia
  • Fourier Series is a periodic function composed of harmonically related sinusoids, combined by a weighted summation. The discrete-time Fourier transform is an example of Fourier series. For functions on unbounded intervals, the analysis and synthesis analogies are Fourier Transform and inverse transform. Fourier Series | Wikipedia
  • Fourier Analysis the study of the way general functions may be represented or approximated by sums of simpler trigonometric functions. Fourier Analysis | Wikipedia

The core formulas. A periodic function with angular frequency ω can be written as f(t) = a0/2 + Σn (an cos nωt + bn sin nωt). For sampled data, the discrete Fourier transform (DFT) of N samples is Xk = Σn=0N-1 xn e-2πikn/N. Computed directly, the DFT takes about N2 operations; the Fast Fourier Transform (popularized by Cooley and Tukey in 1965) takes about N log N, one of the most consequential algorithmic speedups ever found. Fourier thinking also reaches deep into pure mathematics: the Riemann zeta function acts like a frequency analysis of the prime numbers, which is the setting of the quasi-Riemann result described in The Quasi-Riemann Hypothesis.

Why it matters for AI. Fourier ideas appear throughout modern AI in at least six ways:

  • Convolution becomes multiplication. The convolution theorem says convolving two signals in time equals multiplying their spectra in frequency, which allows fast convolutions and explains what convolutional filters do. See Convolution vs. Cross-Correlation (Autocorrelation).
  • Position as frequency. The original Transformer encoded token positions with sines and cosines of many frequencies, and the now-standard rotary position embedding (RoPE) rotates pairs of query and key coordinates by position-dependent angles, a Fourier-style scheme that makes attention depend on relative distance.
  • Mixing tokens with an FFT. Google's FNet replaced self-attention layers with plain Fourier transforms and kept most of BERT's accuracy on the GLUE benchmark while training substantially faster, showing how much "mixing" a parameter-free transform can do.
  • Learning physics in frequency space. Fourier Neural Operators learn mappings between functions, such as from initial conditions to the solution of a fluid-flow equation, by acting on low-frequency Fourier modes. They are widely used for fast surrogate models in simulation, science, and engineering.
  • Fourier features inside language models. Interpretability research in 2024 and 2025 found that LLMs represent numbers with periodic, Fourier-like features and add them using trigonometric structure, as described in Recent Developments.
  • Analog roots. Lord Kelvin's tide-predicting machines of the 1870s summed sinusoids with pulleys and gears, physically performing Fourier synthesis; see Analog Computers. The quantum Fourier transform is the core of Shor's factoring algorithm.

tumblr_mj34wzLpLc1s5nl47o1_r2_400.gif

Load video
YouTube
What is a Fourier Series? (Explained by drawing circles) - Smarter Every Day 205
Doga's a super smart dude who writes a Turkish blog "Bi Lim Ne Güzel Lan" that roughly translates roughly to "Science is Awesome Dude". We had a lot of fun working on this together. He would really appreciate it if you checked out his blog. The fun thing is that most of his articles transcend language. Doga’s Blog (written in Turkish): http://bilimneguzellan.net/ Doga’s original Fourier Series blog article that blew my mind: http://bilimneguzellan.net/fuyye-serisi/ Click here to tweet him "thanks" for http://twitter.com/bilimneguzellan Get a free crate for a kid you love (Awesome Chrsitmas gifts) at: http://www.kiwico.com/smarter Click here if you're interested in subscribing: http://bit.ly/Subscribe2SED Brady’s Video “Optical Tweezers and the 2018 Nobel Prize in Physics - Sixty Symbols” http://www.youtube.com/watch?v=XjXLJMUrNBo

Load video
YouTube
Fourier Transform, Fourier Series, and frequency spectrum
Physics Videos by Eugene Khutoryansky Fourier Series and Fourier Transform with easy to understand 3D animations.

Load video
YouTube
But what is the Fourier Transform? A visual introduction
An animated introduction to the Fourier Transform. Home page: http://www.3blue1brown.com/ Brought to you by you: http://3b1b.co/fourier-thanks Follow-on video about the uncertainty principle: http://youtu.be/MBnnXbOM5S4 Animations largely made using manim, a scrappy open-source Python library. http://github.com/3b1b/manim If you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and has many other quirks you might expect in a library someone wrote with only their own use in mind.

Load video
YouTube
Fourier Analysis For The Rest Of Us
GoldPlatedGoof http://twitter.com/goldplatedgoof

Mathematical Reasoning

When this section was first written in 2023, Large Language Model (LLM)s had limited performance when solving arithmetic reasoning tasks and often provided incorrect answers. Unlike Natural Language Understanding (NLU), math problems typically have a single correct answer, making the task of generating accurate solutions more challenging for LLMs. That picture changed quickly. By 2026, AI systems were scoring perfectly at the International Mathematical Olympiad and contributing to open research problems, as described in Recent Developments, and by October 2026 they were claiming solutions to famous conjectures, as described in What AI Has Given Mathematics. The single-correct-answer property that once made math hard for language models turned out to be the property that made it trainable: when answers can be checked automatically, models can be rewarded for being right. Raw multi-digit arithmetic inside the model is still less reliable than a calculator, which is why production systems routinely hand calculations to code.

Prompting Techniques (2022 to 2023)

Before reasoning models, researchers improved math performance mainly through prompting. These techniques remain useful, and their core ideas (multiple attempts, cross-checking, and explicit algorithms) live on inside today's systems. MathPrompter and algorithmic prompting are two representative examples.

  • For example, MathPrompter is a technique that improves the performance of LLMs on arithmetic problems along with increased reliance in the predictions. MathPrompter uses the Zero-shot Chain of Thought (CoT) prompting technique to generate multiple Algebraic expressions or Python functions to solve the same math problem in different ways and thereby raise the confidence level in the output results. MathPrompter thus leverages solution-verification approaches such as those used by humans (compliance with known results, multi-verification, cross-checking and compute verification) to increase confidence in its generated answers. The MathPrompter pipeline comprises four steps. Given a question:
  1. An algebraic template is generated, replacing the numerical entries with variables;
  2. The LLM is fed multiple math prompts that can solve the generated algebraic expression analytically in different ways;
  3. The analytical solutions are evaluated by allotting multiple random values to the algebraic expression; and
  4. A statistical significance test is applied to the solutions of the analytical functions to find a “consensus” and derive the final solution.
  • Another technique is Algorithmic Prompting, a method of prompt engineering for Large Language Model (LLM)s. The method gives a model a detailed algorithm for solving a math problem. Algorithmic Prompting involves providing a detailed description of the algorithm execution on running examples, and using explicit explanation and natural language instruction to remove ambiguity. The original paper reported error reductions of up to roughly ten times on long addition compared with the best prompting baselines of the time. Using addition as an example, the team behind this technique showed that large language models can apply instructions learned from examples with as few as five digits to problems with as many as 19 digits. This is an example of out-of-distribution generalization and a direct effect of algorithmic prompting.

Reasoning Models and Reinforcement Learning (2024 to 2026)

The big change came from training rather than prompting. Reasoning models generate long internal chains of thought before answering and are trained with reinforcement learning on problems whose final answers, programs, or proofs can be checked automatically, an approach often called reinforcement learning with verifiable rewards. DeepSeek-R1, published openly in early 2025, showed that strong step-by-step reasoning, self-checking, and backtracking can emerge from this kind of training. Several further ingredients matter:

  • Test-time compute. Giving a model more time to think, or sampling many solutions and voting, improves accuracy smoothly; this is the self-consistency idea from MathPrompter, scaled up. Tree of Thoughts (ToT) and related search methods explore branches explicitly.
  • Learned verifiers. Models trained to grade proofs step by step, a form of process supervision that includes the self-verification approach of DeepSeek-Math-V2 (November 2025), let systems improve natural-language proofs where no simple answer key exists.
  • Agents and evolutionary search. Systems such as Google DeepMind's AlphaEvolve and Aletheia wrap models in loops that propose, test, and refine programs, constructions, or arguments, using code execution and literature search as tools. By September 2026 these loops had grown into multi-agent systems at enormous scale, such as the roughly 10,000 coordinating agents OpenAI reported for its Navier-Stokes search.
  • Human collaboration. Many 2026 research results, including the Jacobian conjecture counterexample, came from mathematicians steering models rather than from fully autonomous runs. The First Proof organizers and the Leiden Declaration both stress that the human role should be disclosed.

The critique has also sharpened. Apple's GSM-Symbolic study (2024) showed that accuracy dropped when names, numbers, or irrelevant clauses in grade-school problems were changed, evidence of pattern matching. Later reasoning models narrowed but did not erase such effects, and First Proof showed that confident, wrong proofs remain common. Both the successes and the failures are real, which is why careful evaluation matters so much.

Formal Mathematics and Proof Assistants

A proof assistant such as Lean checks every logical step of a proof mechanically, so a verified proof can be trusted without trusting its author, as long as the theorem statement itself was formalized correctly. AlphaProof learned inside Lean through reinforcement learning. Google DeepMind's Formal Conjectures project states open problems in Lean, and Epoch AI's FrontierMath Erdős requires Lean-verified answers. The main costs have been human effort and library coverage: the 2026 formalization of the Erdős unit distance disproof took about 1.2 million lines of Lean because the deep number theory it relied on had not yet been formalized. That cost fell dramatically in September 2026, when Claude agents formalized all of Fermat's Last Theorem in 11 days, leading Alex Kontorovich to conclude that turning a written human proof into a gap-free Lean proof is now largely solved, even if producing clean, reusable library code is not. "Autoformalization," translating ordinary mathematics into Lean, has therefore shifted from a research frontier to an engineering practice, and the more mathematics is formalized, the cheaper verification becomes for everyone.

How Models Compute: Mechanistic Evidence

Interpretability researchers have started opening the black box on arithmetic. The strongest result so far, Kantamneni and Tegmark's 2025 study, found that mid-sized LLMs encode integers on a generalized helix of periodic components and add them by combining rotations, much as a clock adds hours, a mechanism they confirmed with causal interventions. Competing work finds collections of narrow heuristics instead of one clean algorithm. Both lines of evidence suggest that models build real internal number geometry with finite resolution, which explains both their surprising competence and their occasional strange errors. See Explainable AI for the broader field and Did AI Expand Mathematics or Just Connect the Dots? for how this evidence bears on whether AI understands mathematics.

Load video
YouTube

Math Mistakes | Matt Parker

Load video
YouTube
The Greatest Maths Mistakes | Matt Parker | Talks at Google
When math goes wrong, things can get expensive. Or absolutely hilarious. For this talk we invited YouTube personality (Numberphile, standupmaths), math communicator, comedian, and one third of the Festival of the Spoken Nerd, Matt Parker, to share his favorite math mistakes from his new UK #1 bestseller, "Humble Pi - A Comedy of Maths Errors". Matt exposes errors on the Two Pound Coin, very specific rules for trains operating in Switzerland, and how simple unit conversion slip ups can cost billions of dollars. He also discusses the infamous 256th level of Pac-Man and answers audience questions about more hilarious mathematical failures. Get the book here: http://goo.gl/G4kqw6

Load video
YouTube
What Happens When Maths Goes Wrong? - with Matt Parker
Most of the time, the maths in our everyday lives works quietly behind the scenes, until someone forgets to carry a '1' and a bridge collapses or a plane drops out of the sky. Subscribe for regular science videos: http://bit.ly/RiSubscRibe Matt's book "Humble Pi" available now: https://geni.us/9nPhpn3 Matt Parker is a stand-up comedian and mathematician. He appears regularly on TV and online: as well as being a presenter on the Discovery Channel. His YouTube videos have been viewed over 37 million times. Previously a high-school mathematics teacher, Matt visits schools to talk to students about maths as part of Think Maths and he is involved in the Maths Inspiration shows. In his remaining free time, Matt wrote the books Things To Make and Do in the Fourth Dimension and Humble Pi: A Comedy of Maths Errors. He is also the Public Engagement in Mathematics Fellow at Queen Mary University of London. This talk was filmed in the Ri on 1 March 2019.

Statistics for Intelligence

Statistics is the part of the toolkit that keeps everyone honest. Machine learning itself is statistical estimation at scale, and every claim that one model beats another is a statistical claim. The thesis quote at the top of this page applies most directly here.

Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube
Load video
YouTube

Load video
YouTube


Data Representation

Stem and Leaf Plot

A special table where each data value is split into a "stem" (the first digit or digits) and a "leaf" (usually the last digit). Stem and Leaf Plots | Math is Fun A stem and leaf plot is a great way to organize data by the frequency. It is a great visual that also includes the data. So if needed, you can just take a look to get an idea of the spread of the data or you can use the values to calculate the mean, median or mode. SoftSchools

stem_leaf_plant_img_10.jpg

Load video
YouTube

Histograms

Histograms are one of the most basic statistical tools that we have. They are also one of the most powerful and most frequently used. A histogram groups values into bins and counts how many fall in each, revealing the shape of a distribution: whether it is symmetric or skewed, has one peak or several, or has long tails. In AI work, histograms of weights and activations are a standard way to check whether a model is training healthily and whether it can be quantized safely, since a few extreme outliers can ruin a low-bit format.

Load video
YouTube

Mean, Median, and Mode

  • Mean : The sum of all the data divided by the number of data sets. Example: 8 + 7 + 3 + 9 + 11 + 4 = 42 ÷ 6 = Mean of 7.0
  • Median : The mid data point in a data series organised in sequence. Example : 2 5 7 8 11 14 18 21 22 25 29 (the median is 14, with five data values either side)
  • Mode : The most frequently occurring data value in a series. Example : 2 2 4 4 4 7 9 9 9 9 12 12 13 ( ‘9’ occurs four times, so is the ‘mode’)

The mean is sensitive to outliers and the median is not: add a single value of 1,000 to the median example above and the mean jumps from about 14.7 to about 98, while the median barely moves. That is why typical latency, income, or cost figures are usually reported as medians.

Load video
YouTube

Interquartile Range (IQR)

The interquartile range is a measure of where the “middle fifty” is in a data set. Where a range is a measure of where the beginning and end are in a set, an interquartile range is a measure of where the bulk of the values lie. That’s why it’s preferred over the full range as a measure of spread when reporting things like school performance or SAT scores, since a single extreme value cannot distort it. The interquartile range formula is the first quartile subtracted from the third quartile. A common rule flags values more than 1.5 × IQR beyond the quartiles as potential outliers. Interquartile Range (IQR): What it is and How to Find it | Statistics How To

Load video
YouTube

Box & Whisker Plots (Boxplot)

Presents information from a five-number summary especially useful for indicating whether a distribution is skewed and whether there are potential unusual observations (outliers) in the data set. Box and whisker plots are also very useful when large numbers of observations are involved and when two or more data sets are being compared. Constructing box and whisker plots | Statistics Canada In AI work, side-by-side box plots of scores across random seeds are a quick way to see whether a claimed improvement actually sits inside run-to-run noise.

5214889_01-eng.gif

Load video
YouTube

Standard Deviation

The standard deviation (Greek letter sigma σ for the population standard deviation or the Latin letter s for the sample standard deviation) is a measure that is used to quantify the amount of variation or dispersion of a set of data values. A low standard deviation indicates that the data points tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the data points are spread out over a wider range of values. Standard Deviation | Wikipedia

For a population of N values with mean μ, σ = √(Σ(xi - μ)2 / N). For a sample of n values, s divides by n - 1 instead of n (Bessel's correction), because the sample mean is itself estimated from the same data. The standard deviation of an average of n independent measurements shrinks like σ/√n; this "standard error" is what error bars on benchmark scores are built from. In neural networks, standard deviations also govern weight initialization and layer normalization.

Load video
YouTube
Load video
YouTube

Probability

Probability is the likelihood or chance of an event occurring. When all outcomes are equally likely, probability = (the number of ways of achieving success) ÷ (the total number of possible outcomes). More generally, probabilities are numbers between 0 and 1 assigned to events so that the probabilities of all mutually exclusive possibilities add up to 1. Every modern AI model outputs probabilities. A language model's softmax layer produces a probability distribution over its vocabulary, and "temperature" reshapes that distribution before sampling: low temperature sharpens it toward the most likely token, high temperature flattens it. Diffusion models, Bayesian methods, and reinforcement learning are probability all the way down. 1*J2XZDiZBxiHR5BKTDaX5ZQ.gif

Load video
YouTube
Load video
YouTube
Load video
YouTube

Conditional Probability

The probability of an event ( A ), given that another ( B ) has already occurred: P(A | B) = P(A and B) / P(B). Rearranging gives Bayes rule, P(A | B) = P(B | A) · P(A) / P(B), which tells you how to update a belief after seeing evidence. The same rule underlies naive Bayes classifiers, Bayesian networks, and every method that turns a prior belief and new data into a posterior.

A classic example shows why intuition fails here. Suppose a test catches 99 percent of people with a condition, wrongly flags 1 percent of healthy people, and the condition affects 1 percent of the population. If you test positive, the chance you actually have it is only (0.99 × 0.01) / (0.99 × 0.01 + 0.01 × 0.99) = 50 percent, because healthy people vastly outnumber sick ones. The same base-rate logic explains why an AI classifier with "99 percent accuracy" can still produce mostly false alarms when the thing it hunts for is rare.

Load video
YouTube

Probability Independence

In probability theory, two events are independent, statistically independent, or stochastically independent if the occurrence of one does not affect the probability of occurrence of the other (equivalently, does not affect the odds). Similarly, two random variables are independent if the realization of one does not affect the probability distribution of the other. The concept of independence extends to dealing with collections of more than two events or random variables, in which case the events are pairwise independent if each pair are independent of each other, and the events are mutually independent if each event is independent of each other combination of events. Independence (probability theory) | Wikipedia

Formally, A and B are independent when P(A and B) = P(A) · P(B). Much of machine learning assumes training and test examples are independent and identically distributed (i.i.d.). When that assumption breaks, for instance when the world changes after deployment or when benchmark questions come in related clusters, models and statistics both become unreliable. See Out-of-Distribution (OOD) Generalization and Causation vs. Correlation.

Load video
YouTube

P-Value

YouTube search... ...Google search

In statistics, every conjecture concerning the unknown probability distribution of a collection of random variables representing the observed data X in some study is called a statistical hypothesis. If we state one hypothesis only and the aim of the statistical test is to see whether this hypothesis is tenable, but not, at the same time, to investigate other hypotheses, then such a test is called a significance test. Note that the hypothesis might specify the probability distribution of X precisely, or it might only specify that it belongs to some class of distributions. Often, we reduce the data to a single numerical statistic T whose marginal probability distribution is closely connected to a main question of interest in the study. A statistical hypothesis that refers only to the numerical values of unknown parameters of the distribution of some statistic is called a parametric hypothesis. A hypothesis which specifies the distribution of the statistic uniquely is called simple, otherwise it is called composite. Methods of verifying statistical hypotheses are called statistical tests. Tests of parametric hypotheses are called parametric tests. We can likewise also have non-parametric hypotheses and non-parametric tests. The p-value is used in the context of null hypothesis testing in order to quantify the idea of statistical significance of evidence, the evidence being the observed value of the chosen statistic T. Null hypothesis testing is a reductio ad absurdum argument adapted to statistics. In essence, a claim is assumed valid if its counterclaim is highly implausible. p-value | Wikipedia

In symbols, for a test statistic T with observed value t, the one-sided p-value is P(T ≥ t, assuming the null hypothesis is true). The American Statistical Association's 2016 statement lists the most common misreadings. A p-value can show how incompatible the data are with a specified model, but it does not measure the probability that the hypothesis is true, does not measure the size or importance of an effect, and should never be the sole basis for a conclusion. Proper inference requires full reporting, including every test that was run. That last point matters for AI: if a team tries twenty prompt variants and reports the best one, a "significant" improvement at p < 0.05 is expected by chance alone, and the same logic applies when a lab attempts thousands of open problems and publishes only its successes. Corrections for multiple comparisons, pre-registered evaluation plans, held-out test sets, and disclosure of failed attempts are the defenses.

Load video
YouTube
StatQuest: P Values, clearly explained / How to calculate p-values
StatQuest with Josh Starmer People often confuse p-values with probabilities. In this StatQuest we learn how to calculate p-values using both discrete data (like coin tosses) and continuous data (like height measurements). At the end, we explain the differences between 1 and 2-sided p-values and why you should avoid 1-sided p-values if possible. For a complete index of all the StatQuest videos, check out: https://statquest.org/video-index/

Confidence Interval (CI)

YouTube search... ...Google search

In statistics, a confidence interval (CI) is a type of estimate computed from the statistics of the observed data. This proposes a range of plausible values for an unknown parameter (for example, the mean). The interval has an associated confidence level that describes how reliably the method captures the true parameter. Before the data are collected, a valid procedure with confidence level gamma has a probability gamma of producing an interval that contains the true underlying parameter; once a particular interval has been computed, it either contains the parameter or it does not. The level of confidence can be chosen by the investigator. In general terms, a confidence interval for an unknown parameter is based on sampling the distribution of a corresponding estimator. More strictly speaking, the confidence level represents the frequency (i.e. the proportion) of possible confidence intervals that contain the true value of the unknown population parameter. In other words, if confidence intervals are constructed using a given confidence level from an infinite number of independent sample statistics, the proportion of those intervals that contain the true value of the parameter will be equal to the confidence level. For example, if the confidence level (CL) is 90% then in a hypothetical indefinite data collection, in 90% of the samples the interval estimate will contain the population parameter. The confidence level is designated before examining the data. Most commonly, a 95% confidence level is used. However, confidence levels of 90% and 99% are also often used in analysis. Factors affecting the width of the confidence interval include the size of the sample, the confidence level, and the variability in the sample. A larger sample will tend to produce a better estimate of the population parameter, when all other factors are equal. A higher confidence level will tend to produce a broader confidence interval. The most common recipe for a mean is x̄ ± z · s/√n, where x̄ is the sample mean, s the sample standard deviation, n the sample size, and z about 1.96 for 95 percent confidence. Because the width shrinks only like 1/√n, halving an error bar requires four times as much data.

Load video
YouTube
Understanding Confidence Intervals: Statistics Help
Dr. Nic's Maths and Stats This short video gives an explanation of the concept of confidence intervals, with helpful diagrams and examples. A good follow-up to check understanding is the video: Confidence Intervals - a quiz to develop understanding. https://youtu.be/gvVD-xlY2Hc See http://creativemaths.net/videos/ for all of Dr. Nic's videos organized by topic.

Load video
YouTube
StatQuest: Confidence Intervals
StatQuest with Josh Starmer A StatQuest http://statquest.org/ for Confidence Intervals. For a complete index of all the StatQuest videos, check out: http://statquest.org/video-index/

Statistics in AI Evaluation

Benchmarks are experiments, and they deserve the same statistics as any other experiment, which is the core argument of the model evaluation literature. Worked example: how precise is a 43-problem benchmark? FrontierMath Tier 4 (v2) has 43 problems. A model scoring 83 percent has a binomial standard error of about √(0.83 × 0.17 / 43) ≈ 0.057, so a 95 percent confidence interval runs roughly from 72 to 94 percent. Near 100 percent the simple formula breaks down, and the Wilson interval is better: for 42 of 43 problems solved (about 97.7 percent), the Wilson 95 percent interval is roughly 88 to 99.6 percent. Two models scoring around 83 and 88 percent on this set are not clearly different from these numbers alone.

Better practices, following Miller's recommendations:

  • Report standard errors alongside every score, as other sciences do.
  • Use paired comparisons. When two models answer the same questions, analyze the per-question differences. Pairing removes the variation due to question difficulty and often detects differences that unpaired intervals would miss.
  • Cluster when questions are related. Translations of one question, or several questions about the same passage, are not independent. Treating them as independent makes error bars too narrow.
  • Reduce noise at the source. Average several samples per question, or use the model's probability of the correct answer instead of a single right-or-wrong grade.
  • Plan sample size in advance. Power analysis tells you how many questions you need to detect the improvement you care about.
  • Account for flawed items. FrontierMath's June 2026 audit corrected or removed problems with errors in 42 percent of items; label noise of that size can swamp model differences.
  • Check for contamination. Problems whose solutions appear online before a model's training cutoff can inflate scores. FrontierMath Erdős handles this by filtering out problems solved before each model's cutoff.
  • Report the denominator. A release of a few hundred solved problems means something very different if it came from 400 attempts or from 4,000, so the number of problems tried is as important as the number solved.
  • Beware threshold metrics. Researchers have shown that some apparent "emergent" jumps in capability come from all-or-nothing scoring rules, and that smoother metrics reveal gradual improvement. See Emergence.

Regression

Regression is a statistical technique for estimating the relationships among variables. Regression | Wikipedia The simplest case, linear regression, fits y ≈ Xβ by minimizing the sum of squared errors, which has the closed-form solution β = (XTX)-1XTy. That one formula ties together nearly everything on this page: it is linear algebra (matrix products and an inverse), calculus (setting the gradient of the loss to zero), and statistics (under Gaussian noise, least squares is maximum likelihood). Logistic regression adds a sigmoid to predict probabilities, and a neural network can be viewed as regression with learned, layered features. Regression also carries the classic warning that correlation in data does not establish causation; see Causation vs. Correlation.


The Physics

AI math is not abstract once it runs. Every multiply, add, and memory read costs time and energy, and the physics of computation explains why so much research focuses on doing fewer, cheaper, or more local operations.

Counting operations. Multiplying an m×k matrix by a k×n matrix takes mkn multiplications and roughly 2mkn floating-point operations (FLOPs). A Transformer forward pass costs about 2N FLOPs per token for N parameters, and training costs about 6N FLOPs per token, so total training compute is roughly 6ND for D training tokens. A hypothetical 70-billion-parameter model trained on 15 trillion tokens therefore needs about 6 × 7×1010 × 1.5×1013 ≈ 6.3×1024 FLOPs, on the order of four million accelerator-hours at a sustained 400 teraFLOP/s each. Matrix multiplication dominates that bill.

Energy per operation. Mark Horowitz's widely cited ISSCC 2014 keynote, "Computing's Energy Problem," measured these costs in a 45 nm process. Newer chips are more efficient in absolute terms, but the ratios still hold: moving data costs far more than computing on it, and multiplying costs more than adding.

Operation (45 nm) Energy (picojoules) Relative to a 32-bit float add
8-bit integer add 0.03 about 0.03x
32-bit integer add 0.1 about 0.1x
8-bit integer multiply 0.2 about 0.2x
16-bit float add 0.4 about 0.4x
32-bit float add 0.9 1x
16-bit float multiply 1.1 about 1.2x
32-bit integer multiply 3.1 about 3.4x
32-bit float multiply 3.7 about 4x
32-bit read from 8 KB SRAM 5 about 6x
32-bit read from off-chip DRAM 640 about 700x

Two lessons fall straight out of the table. Replacing multiplies with adds (the idea behind ternary and MatMul-free models) saves real energy. Keeping data close to the arithmetic (the idea behind in-memory and analog computing) saves even more.

The memory wall and arithmetic intensity. The roofline model says a chip's attainable speed is the smaller of its peak FLOP rate and its memory bandwidth times the arithmetic intensity (FLOPs performed per byte moved). Generating one token at batch size one touches every weight once and does only about two FLOPs per weight, so inference is usually limited by memory bandwidth. Streaming 140 GB of 16-bit weights for a 70-billion-parameter model at about 3 TB/s takes roughly 47 ms per token, no matter how fast the arithmetic units are; the same weights stored in 4 bits take about 12 ms. That is why quantization speeds up inference even when the math is unchanged.

Number formats. Neural networks tolerate surprisingly low precision because training is already noisy. Fewer bits per number also means less memory traffic, which the roofline argument above shows is usually the real bottleneck. The trend has therefore been steadily downward:

Format Bits (sign, exponent, mantissa) Typical AI use
FP32 1, 8, 23 Master weights, optimizer state, accumulation
BF16 1, 8, 7 Default training format on modern accelerators
FP16 1, 5, 10 Inference and older mixed-precision training
FP8 (E4M3 or E5M2) 1, 4, 3 or 1, 5, 2 Matrix multiplies for training and inference on recent GPUs
FP4 (E2M1) with block scales 1, 2, 1 plus a shared scale per block MXFP4 (blocks of 32) and NVFP4 (blocks of 16) for inference and, since 2025, pretraining
Ternary about 1.58 bits per weight BitNet b1.58 and MatMul-free models

A ternary weight takes one of three values {-1, 0, +1}, so it carries log23 ≈ 1.58 bits of information. A dot product with ternary weights needs no multiplications at all: add the activations whose weight is +1, subtract those whose weight is -1, and skip the zeros. That is why ternary models can run on hardware built mostly from adders.

How far physics allows us to go. Landauer's principle sets the minimum energy to erase one bit at kBT ln 2 ≈ 2.9×10-21 joules at room temperature. A 32-bit float add at about 0.9 pJ is roughly a hundred million times above that limit. Physics leaves enormous headroom; the practical limits are wires, memory traffic, and heat.

Analog physics does math for free. In a resistive crossbar, Ohm's law (I = G·V) multiplies each input voltage by a stored conductance, and Kirchhoff's current law sums the resulting currents on a shared wire. A whole matrix-vector product happens in one physical step, an idea closely related to neuromorphic computing. The catch is that analog devices are noisy, drift over time, hold only a few bits of effective precision, and need analog-to-digital converters that can eat much of the energy saved. Nonvolatile devices are also slow and wear out when rewritten often, which is why the attention cache, rewritten at every generated token, has been the hardest part of a Transformer to move into analog hardware. See Analog Computers.

Fast algorithms have numerical side effects. Strassen-style algorithms trade multiplications for extra additions, which increases memory traffic and amplifies rounding error. On GPUs, tensor cores make the standard algorithm so efficient that fast algorithms help only for large matrices or special arithmetic. Reducing the multiplication count is a mathematical win that does not automatically become a hardware win.

Books

61JTiB3rHQL._SX394_BO1,204,203,200_.jpg 61wQSjN0MEL._SX348_BO1,204,203,200_.jpg

Analog Computers

YouTube ... Quora ...Google search ...Google News ...Bing News


Substitute brass for brains - Lord Kelvin; William Thomson, 1st Baron

An analog computer represents numbers as physical quantities, such as the angle of a shaft, the height of water, or a voltage, and lets physics do the arithmetic. Gears multiply by constant ratios, differential gears add, and integrators accumulate. For centuries these were the most powerful computers available, from the Antikythera mechanism, an ancient Greek gear train that modeled astronomical cycles, to naval fire-control computers in the Second World War. Digital computers replaced them because digital signals can be copied and corrected without accumulating noise. AI is reviving the analog idea, because the matrix-vector products at the heart of neural networks map naturally onto physics and tolerate some noise.

Mechanical Integrator

The key component of mechanical analog computers is the integrator. In the wheel-and-disc version, a flat disc turns with an input quantity x, and a small wheel of radius a rests on the disc at an adjustable distance r(t) from its center. The point of contact moves with speed proportional to r, so the wheel turns by dφ = (r/a) dx. Add up all those small turns and the wheel's total rotation is φ = (1/a)∫ r dx: the machine has computed an integral. James Thomson, Lord Kelvin's brother, described the disc-ball-cylinder integrator in 1876, and Kelvin used integrators in his harmonic analyzer, which extracted the Fourier coefficients of measured tide records. His tide-predicting machines then ran the process in reverse, summing a dozen or more sinusoids with pulleys and a single wire to draw future tides, which was literally a mechanical Fourier synthesis. In 1931, Vannevar Bush's differential analyzer at MIT chained integrators together to solve differential equations, a direct ancestor of both electronic computing and today's interest in physical computation.

Analog Computing Returns for AI

Modern analog AI hardware uses the physics described in The Physics: store each weight as a conductance in a memory cell, apply inputs as voltages, and read the sum of currents on each column as one entry of a matrix-vector product. IBM has demonstrated chips with millions of phase-change memory devices running speech recognition and has mapped Transformer layers with static weights onto analog cores. The attention cache was the hard part, because it changes at every generated token and nonvolatile memories are slow and wear out under constant rewriting. A 2025 Nature Computational Science design addressed this with "gain cells," small volatile capacitors that can be rewritten quickly and compute dot products in analog form, using charge-to-pulse circuits to avoid power-hungry analog-to-digital converters. The capacitors' natural leakage makes older tokens fade, which the authors related to known attention-decay schemes, and they used sliding-window attention plus an adaptation algorithm to reach GPT-2-level text performance without training from scratch. Their estimates show up to five orders of magnitude lower attention energy and two orders lower latency than GPUs. Those are architecture and circuit estimates, not measurements of a complete fabricated system, and analog approaches still face noise, drift, limited precision, and converter overhead. The digital alternative, ternary and MatMul-free models, chases the same goal from the other direction: make the arithmetic so simple that it barely costs anything.

Load video
YouTube
The Mechanical Integrator - a machine that does calculus
This video explains the function of the mechanical integrator, a mechanism crucial to the development of mechanical analog computers throughout the twentieth century. This video is part of a project I have been working on in collaboration with Professor Michael Littman of Princeton University. One of the goals for developing this specific machine was to use it as a supplementary tool when teaching calculus and differential equations, so I made this video to demonstrate the machine's function with specific emphasis on its connection to calculus. - Jack Monaco

Load video
YouTube
The Most Powerful Computers You've Never Heard Of
Analog computers were the most powerful computers for thousands of years, relegated to obscurity by the digital revolution. This video is sponsored by Brilliant.

Load video
YouTube
How did the Enigma Machine work?
Jared Owen... Let's use 3D animation to go inside the Enigma Machine!

Load video
YouTube
I Made A Water Computer And It Actually Works
Steve Mould... Computers add numbers together using logic gates built out of transistors. But they don't have to be! They can be built out of greedy cup siphons instead! I used specially designed siphones to works as XOR and AND gates and chained them together so they add 4 digit binary numbers. (Note that this water computer is digital, not analog: each siphon is either full or empty, just like a transistor is either on or off.)

Retrieved from "https://primo.ai/w/index.php?title=Mathematics&oldid=50015"