History of Artificial Intelligence (AI)

From PRIMO.ai
Jump to navigation Jump to search

YouTube ... Quora ...Google search ...Google News ...Bing News


Today, " Artificial Intelligence (AI)" is a ubiquitous term used by researchers, corporations, and consumers to describe software systems capable of performing complex tasks that historically required human cognition. In technical practice, the history of AI is the seventy-year effort to determine how knowledge, perception, deduction, and planning can be formalized mathematically and executed on physical hardware.

The field grew out of a mid-twentieth-century convergence of mathematical logic, information theory, neurobiology, and statistical physics. Early architects including Alan Turing, Warren McCulloch, Walter Pitts, Claude Shannon, John McCarthy, Marvin Minsky, Nathaniel Rochester, Herbert Simon, and Allen Newell sought to answer a foundational question: can human thought be reduced to formal rules manipulating discrete symbols, or does intelligence emerge from distributed networks of simple units adjusting their connections through experience?

That question divided AI research into two competing traditions for more than four decades. The symbolic tradition, often called Good Old-Fashioned AI (GOFAI), treated the mind as a logical calculus. Programmers explicitly wrote rules and ontologies to guide heuristic searches through state spaces. By contrast, the connectionist tradition, pioneered by Frank Rosenblatt and later advanced by John Hopfield, Geoffrey Hinton, Yann LeCun, and Yoshua Bengio, modeled intelligence on simplified biological neural networks and statistical mechanics. Connectionists argued that engineers should not hand-code domain rules; instead, algorithms should learn internal representations directly from raw data by minimizing an objective error function.

Across seventy years of computer science, methods that leverage general-purpose learning and scalable computation (specifically search and gradient-based learning) have consistently outperformed methods that rely on human experts manually encoding domain knowledge. While early symbolic systems excelled in narrow, closed worlds with clean rules, they shattered when confronted with the noisy, high-dimensional ambiguity of natural language, vision, and physical environments. Modern Deep Learning, Generative AI, and autonomous reasoning agents succeeded by replacing brittle human-written heuristics with massive parameter spaces trained across trillions of data tokens and optimized through Reinforcement Learning.


The seven-decade evolution of Artificial Intelligence represents a fundamental epistemological shift from manually encoding human knowledge into rigid symbolic rules toward engineering statistical, connectionist, and reinforcement-driven architectures that autonomously discover representations, scale across pre-training and test-time compute, and execute multi-step reasoning, while continuing to face foundational constraints in formal verification, sample efficiency, and out-of-distribution generalization.


Core Technical Foundations: Mechanism and Significance

The Mechanism behind artificial intelligence has evolved through four major generations. Over time, the main job of solving problems has shifted away from human programmers writing step-by-step instructions to computers learning patterns and figuring out solutions on their own.

  1. Rule-Based Systems and Smart Search (First Generation): In the earliest days of AI, programmers broke every problem down into a map of possible steps and choices. Human experts wrote out lists of facts and strict "IF-THEN" rules for the computer to follow. The computer then used these rules to work forward from known facts to a conclusion, or backward from a goal to see what steps were needed. In complex tasks like chess, checking every possible future move creates far too many options for any computer to calculate. To handle this, programmers created shortcuts (called heuristics) that helped the computer ignore bad moves and focus only on promising ones. The biggest weakness of this approach is that humans simply cannot write down every common-sense rule needed to recognize a face or understand a casual conversation.
  2. Statistical Learning and Human-Chosen Clues (Second Generation): Instead of writing strict rules, the second generation of Machine Learning (ML) taught computers to make predictions by finding statistical patterns in data. However, human engineers still had to decide which clues (called features) the computer should look at, such as counting specific words in an email or measuring edges in a photo. Once humans picked the clues, an algorithm such as a Support Vector Machine (SVM) or a Random Forest learned how to draw a line separating one category from another, like spam versus regular email. While these systems handled messy real-world data much better than strict rules, they were still limited by how well humans chose the starting clues.
  3. Deep Learning and Automatic Pattern Discovery (Third Generation): Deep Learning removed the need for humans to hand-pick clues by using artificial neural networks made of many connected layers. When raw information (like pixels in a picture, sound waves, or words) passes through the network, the early layers spot simple patterns like edges or word parts, and the deeper layers combine them to recognize whole objects or complex ideas. During training, the computer compares its guess to the correct answer and measures the size of its mistake. Using a feedback process called backpropagation (popularized for neural networks by David Rumelhart, Geoffrey Hinton, and Ronald Williams in 1986), the system works backward through the layers and slightly adjusts millions or trillions of internal connections so it makes a better guess next time. In 2017, a design called the Transformer replaced older networks that read words one at a time. Using a technique called self-attention, a Transformer looks at all the words in a passage at once and sees how they connect to one another. This breakthrough allowed companies to train massive models on text from across the internet by having them predict the next word in a sentence, and then refine them with human feedback to act as helpful assistants.
  4. Thinking Before Answering and AI Agents (Fourth Generation): Standard Large Language Models (LLMs) generate answers one word at a time on the first try, spending the exact same effort per word whether answering an easy trivia question or solving a hard puzzle. Fourth-generation reasoning models and AI agents work differently by taking extra time to "think" before giving a final answer. The model writes out an internal step-by-step scratchpad (called a Chain of Thought), tests different ideas, catches its own mistakes, and backs up when it hits a dead end. These systems can also act as agents that use outside tools, such as running computer code or searching the web, to check their work. Engineers train these models by rewarding them when their final answers pass strict tests (like a math proof checking out or software code running without bugs) and by scoring each step of their reasoning.
Comparison of the Four Technological Generations of Artificial Intelligence
Generation & Era Core Approach Human Engineer's Role Machine's Role Main Limitation Key Examples & Milestones
First Generation
(1956 - late 1980s)		 Symbolic AI & Rule-Based Expert Systems Writes both the search rules and all the "IF-THEN" facts Follows logical rules and searches through possible moves Too many possible moves to check, and humans cannot write rules for common sense Logic Theorist (1956), ELIZA (1966), SHRDLU (1970), MYCIN (1974), XCON (1980)
Second Generation
(late 1980s - 2011)		 Statistical Machine Learning (ML) Chooses which data clues (features) the computer should study and sets the goal Learns the best way to sort data or make predictions using those clues Relies on humans picking the right clues; struggles with raw photos, audio, and video Support Vector Machines (1995), IBM Deep Blue (1997), LeNet-5 (1998), Random Forests (2001)
Third Generation
(2012 - 2024)		 Deep Learning, Transformers & Generative AI Designs the neural network structure and prepares large training datasets Discovers its own patterns and makes predictions directly from raw data Requires massive computer power and memory; can make up false answers on the first try AlexNet (2012), AlphaGo (2016), Transformer (2017), GPT-3 (2020), AlphaFold2 (2020), ChatGPT (2022)
Fourth Generation
(2024 - Present)		 Reasoning Models & Autonomous AI Agents Sets up testing environments, safety rules, and outside tools Takes extra time to think step-by-step, fixes its own errors, and uses software tools Long thinking steps use huge amounts of computer memory and can still fail on unfamiliar problems OpenAI o1/o3 (2024-2025), DeepSeek-R1 (2025), Claude 3.7/4 Agentic Systems, Gemini Deep Research

A major question still remains about how these systems actually work: are they truly learning how to reason, or are they just blending together patterns from their training data? Because neural networks rely on statistics and probabilities rather than strict logical rules, their step-by-step thinking can still include made-up facts (hallucinations) or break down when a problem looks too different from what they have seen before.


Why This Matters

Why This Matters reaches far beyond computer science, changing how scientists make discoveries, how engineers build software, and how we define intelligence itself.

- Speeding Up Scientific Discovery: For its first fifty years, researchers mostly tested AI on simple puzzles and board games. In the 2020s, deep learning became a powerful tool for real-world biology and physics. The biggest breakthrough came with Google DeepMind's AlphaFold2 (2020) and AlphaFold3 (2024), which solved a fifty-year-old biology mystery: predicting the exact 3D shape of a protein just from its list of chemical building blocks. This achievement won Demis Hassabis and John Jumper (along with David Baker for protein design) the 2024 Nobel Prize in Chemistry. Today, similar AI tools help scientists study chemistry, control super-heated plasma in fusion reactors, speed up Molecular drug discovery, map brain connections in Neuroscience, and do non-destructive Using AI to reveal historical mysteries (such as reading ancient scrolls burned by Mount Vesuvius without unrolling them) as well as sorting fossils in Paleontology. - Changing Software Engineering and Game Strategy: Across the tech industry, AI agents and large models are changing how people write and test computer code. In Gaming, Game Theory, and Design, AI grew from simple programs that looked a few moves ahead in games where both players see the whole board (like Checkers and Chess) to advanced systems that combine deep learning with smart search in massive games (like AlphaGo and AlphaZero). Newer systems even master games with hidden information and multiple players, such as Libratus and Pluribus in poker, AlphaStar in StarCraft II, and Generative AI characters in video games. - Rethinking Intelligence and How We Test It: The history of AI shows a pattern known as the "AI Effect" (often summed up as: "AI is whatever hasn't been done yet"). Whenever a computer masters a skill that people used to think required human intelligence—like proving math theorems, beating world chess champions, translating languages, or passing medical and law exams—people often brush it off as "just math" rather than real intelligence. At the same time, modern Conversational AI chatbots sound so natural that the famous 1950 Turing Test (which asks if a machine can fool a human in conversation) is no longer enough. Because humans are easily fooled by polite, confident writing, researchers now use tougher tests that measure logic, real-world coding (SWE-bench), and the ability to solve brand-new visual puzzles without memorizing answers (such as the Abstraction and Reasoning Corpus, or ARC-AGI). - Key Limits and Open Questions: If scaling up current AI models is enough to reach human-level intelligence, then scientific progress and economic growth will depend heavily on how much computer power and electricity a country can build. However, big technical hurdles remain. Deep neural networks often forget old skills when learning new ones, need millions more examples to learn than a human child does, lack a built-in common-sense understanding of the physical world, and cannot guarantee that their answers are 100% safe or accurate. Whether solving these problems simply requires more computing power and better world models, or requires a fifth generation of AI that combines neural networks with strict logical rules (neuro-symbolic AI), is one of the biggest open questions in science today.

The Turk

- The Turk

- Analog Computers

Long before electronic digital computers existed, eighteenth- and nineteenth-century inventors built intricate clockwork automata that forced philosophers and engineers to confront the nature of mechanized thought. The most famous and consequential of these devices was The Turk (also known as the Mechanical Turk or Automaton Chess Player), constructed in 1769 by Hungarian-born engineer and court counselor Baron Wolfgang von Kempelen (1734-1804) to impress Empress Maria Theresa of Austria.

Externally, The Turk consisted of a life-sized carved wooden mannequin dressed in Ottoman robes and a turban, seated behind a large maple cabinet measuring roughly 3.5 feet long, 2 feet wide, and 2.5 feet high. Before each match, Kempelen opened the cabinet doors in a specific sequence and held a candle inside, revealing an intricate tangle of brass gears, cogs, levers, and clockwork cylinders designed to convince spectators that the cabinet housed a purely mechanical calculating machine. In reality, The Turk was an elaborate engineering illusion: a sliding seat, false mechanical components, pantograph linkage arms, and magnetic chess pieces allowed a concealed human chess master inside the cabinet to track the opponent's moves from beneath the board and operate the mannequin's left arm.

Despite being a hoax, The Turk played a pivotal role in the prehistory of artificial intelligence for three reasons:

  1. Stimulating Early Computer Architecture: For eighty-five years, under Kempelen and later showman Johann Nepomuk Mälzel, The Turk toured Europe and the Americas, defeating prominent historical figures including Benjamin Franklin and Napoleon Bonaparte. In 1819, British mathematician Charles Babbage played against The Turk twice in London. Although Babbage correctly suspected human agency inside the cabinet, the spectacle of mechanized decision-making directly informed his mathematical exploration of game-playing algorithms and his subsequent designs for the Difference Engine and the Analytical Engine, for which Ada Lovelace wrote the 1843 notes analyzing whether a machine could ever "originate anything" beyond what programmers order it to perform.
  2. Philosophical Analysis of Determinism vs. Agency: In an influential 1836 essay (*Maelzel's Chess-Player*), Edgar Allan Poe analyzed The Turk and argued that it could not be a pure machine because a true machine would always operate deterministically and win every game without temporal hesitation. While Poe's assumption that machines must be strictly deterministic was later overturned by stochastic computing and heuristic search, his essay initiated public debate over the behavioral criteria separating human cognition from mechanical execution.
  3. The Lineage of "Artificial Artificial Intelligence" in Modern ML: In 2005, Amazon launched Amazon Mechanical Turk (MTurk), a crowdsourcing marketplace explicitly named after Kempelen's automaton. MTurk inverted the eighteenth-century illusion into a digital infrastructure: software APIs dispatch tasks that remain difficult for computers to a distributed global workforce of human annotators sitting behind screens. Ironically, the modern Deep Learning and Generative AI revolutions depended directly on this "artificial artificial intelligence." Landmark datasets such as ImageNet (which catalyzed the 2012 deep learning breakthrough via over 14 million human-labeled images crowdsourced on MTurk) and the human preference annotation pipelines used for Reinforcement Learning from Human Feedback (RLHF) in ChatGPT, Claude, and Gemini rely on hidden human labor training and correcting statistical models behind the curtain.

truco-Turco-ROS-821x335.png

Load video
YouTube
Mechanical Marvels - Automaton: The Chess Player "Android," 1769
Touted as an android that could defeat chess masters, Wolfgang von Kempelen's famed illusion debuted at the court of Empress Maria Theresa during wedding celebrations for her daughter in 1769. Over the course of the eighteenth century, the chess player (known in its time as The Turk for its costume) won games against Catherine the Great and Benjamin Franklin. When Napoléon Bonaparte tried to cheat, it wiped all the pieces from the board. The mysterious machine sparked discussions of the possibilities and limits of artificial intelligence, and it inspired the development of the power loom, the telephone, and the computer. The original and its secrets were destroyed in a fire in 1854. The subject of more than eight hundred publications attempting to uncover its secrets, Kempelen's illusion also inspired a 1927 silent movie, The Chess Player, directed by Raymond Bernard. In the sequence shown here, the inventor presents his creation at court. The year of its release, this early science-fiction drama attracted more attention than Fritz Lang's Metropolis, a now-legendary film that also involves an android. Featured Artwork: The Chess Player (The Turk), Original ca. 1769. Wolfgang von Kempelen (1734-1804). Austrian, Vienna. Wood, brass, fabric, steel. Collection of Mr. John Gaughan, Los Angeles

Chronological Eras & Generations of AI

Before AI became a unified field of study, the discipline lacked a standardized name or distinct identity, and its evolution has since been defined by foundational theoretical precursors followed by four distinct technological generations.

Competing Visions Before 1956

During the 1940s and early 1950s, mathematicians, logicians, neurophysiologists, and engineers laid the mathematical groundwork for machine intelligence, categorizing their research under several competing frameworks:

- Mathematical Logic and the Universal Computing Machine (1936-1950): In 1936, British mathematician Alan Turing introduced the universal Turing machine, proving that any Formal mathematical procedure could be executed by a machine manipulating binary symbols on a tape. In his landmark 1950 paper in the philosophy journal *Mind*, titled *Computing Machinery and Intelligence*, Turing bypassed metaphysical debates over whether machines could "think" by proposing an operational behavioral evaluation he called the "Imitation Game," now universally known as the Turing Test. In the same paper, Turing also anticipated machine learning, proposing that rather than building a complete adult mind by hand, researchers might program a "child machine" and subject it to an educational process of rewards and punishments. - Early Mathematical Connectionism (1943-1949): In 1943, neurophysiologist Warren McCulloch and logician Walter Pitts published *A Logical Calculus of the Ideas Immanent in Nervous Activity*, demonstrating that networks of simplified binary threshold artificial neurons could compute any logical proposition (`AND`, `OR`, `NOT`). In 1949, psychologist Donald Hebb published *The Organization of Behavior*, proposing a physiological learning rule (Hebbian plasticity) in which synaptic strength increases when two connected neurons fire simultaneously ("neurons that fire together, wire together"). In 1951, Marvin Minsky and Dean Edmonds constructed the SNARC (Stochastic Neural Analog Reinforcement Calculator), the first hardware artificial neural network, using 3,000 vacuum tubes and a B-24 bomber autopilot mechanism to simulate a rat learning to navigate a maze. - Cybernetics: Formulated by MIT mathematician Norbert Wiener in 1948 and popularized during a major 1951 conference in Paris, cybernetics focused on continuous feedback loops, homeostasis, communication, and analog control in both machines and living organisms (closely tied to Bio-inspired Computing and Analog Computers). - Automata Studies: Centered on discrete self-operating machines, finite-state automata, and information theory, this phrase was favored by Claude Shannon, the "father of information theory" who in 1950 also published the first foundational scientific paper on programming a computer to play chess (*Programming a Computer for Playing Chess*), introducing minimax search over heuristic evaluation functions. - Complex Information Processing: Computer science pioneers Herbert Simon, Allen Newell, and Cliff Shaw at the RAND Corporation and Carnegie Institute of Technology used this descriptor to define their early computational and cognitive research into how human problem-solving could be modeled using symbol list processing.

The Birth of AI: The Dartmouth Workshop (1956)

The need for a distinct, unifying identity culminated in an August 31, 1955 proposal for a two-month, ten-man summer study at Dartmouth College in Hanover, New Hampshire, drafted by mathematician John McCarthy alongside Marvin Minsky, Nathaniel Rochester (architect of the IBM 701), and Claude Shannon. Seeking to separate digital symbolic computation from Norbert Wiener's analog control theory in cybernetics and avoid the narrow mathematical connotations of automata studies, McCarthy coined the phrase "Artificial Intelligence."

Held over roughly six to eight weeks in the summer of 1956, the Dartmouth Summer Research Project on Artificial Intelligence formally established AI as an independent scientific discipline. Attendees included Trenchard More, Arthur Samuel, Oliver Selfridge, Ray Solomonoff, Allen Newell, and Herbert Simon. During the workshop, Newell and Simon unveiled the Logic Theorist, a program written in their Information Processing Language (IPL) that successfully proved 38 of the first 52 theorems in Chapter 2 of Alfred North Whitehead and Bertrand Russell's *Principia Mathematica*, even finding a more elegant proof for Theorem 2.85 than Russell and Whitehead had published.

The Dartmouth Workshop was anchored by a bold foundational conjecture written in the 1955 proposal that continues to frame the discipline: every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. The proposal outlined seven key research problems that remain central to AI: automatic computers, programming a computer to use a language, neuron nets, theory of the size of a calculation (computational complexity), self-improvement, abstractions, and randomness and creativity.

First Generation AI: Good Old-Fashioned AI & The AI Winters

The initial decades following Dartmouth were dominated by First Generation AI, later termed Good Old-Fashioned AI (GOFAI) by philosopher John Haugeland in 1985. Grounded in Newell and Simon's Physical Symbol System Hypothesis (which stated that a physical symbol system has the necessary and sufficient means for general intelligent action), programmers handcrafted explicit symbolic representations and search rules. While early textbook examples performed narrow operations like sorting data or arithmetic, flagship First Generation programs tackled ambitious symbolic reasoning tasks: Newell and Simon's *General Problem Solver* (1957), McCarthy's invention of the LISP programming language (1958), Arthur Samuel's checkers program at IBM (which demonstrated early reinforcement learning via self-play and defeated a state champion in 1962), Joseph Weizenbaum's pattern-matching psychotherapist chatbot *ELIZA* (1966), and Terry Winograd's *SHRDLU* (1970), which manipulated and conversed about a virtual "blocks world."

Meanwhile, the competing connectionist approach achieved early momentum when Cornell psychologist Frank Rosenblatt built the Mark I Perceptron (1957-1958), a single-layer neural network hardware device for visual pattern recognition. However, in 1969, Marvin Minsky and Seymour Papert published their book *Perceptrons*, proving mathematically that single-layer perceptrons were limited to linearly separable functions and could not even compute the simple two-input exclusive-OR (`XOR`) logical function. Because no general algorithm was yet widely known for training multi-layer networks, funding for connectionist Neural Network research largely evaporated for over a decade.

Symbolic GOFAI soon encountered its own physical and mathematical wall. Programs that worked brilliantly in "micro-worlds" with a dozen blocks failed catastrophically on real-world problems due to combinatorial explosion, the lack of commonsense world knowledge, and the inability of pure logic to handle uncertainty. When early machine translation projects failed to deliver after heavy Cold War investment (documented in the 1966 ALPAC report) and Sir James Lighthill issued his highly critical 1973 report to the British Science Research Council (*Artificial Intelligence: A General Survey*), both the UK government and the U.S. Defense Advanced Research Projects Agency (DARPA) slashed academic AI grants, triggering the First AI Winter (roughly 1974-1980).

In the 1980s, First Generation symbolic AI experienced a major commercial resurgence through Expert Systems: programs designed to solve narrow, high-value domain problems by reasoning through knowledge bases of thousands of handcrafted `IF-THEN` rules elicited from human specialists. Pioneered by Edward Feigenbaum's DENDRAL (for chemical mass spectrometry) and MYCIN (for diagnosing blood infections), and commercialized through Carnegie Mellon's XCON/R1 system for Digital Equipment Corporation (which saved DEC an estimated $40 million annually configuring VAX computer orders), expert systems sparked a global corporate boom and inspired Japan's $400 million Fifth Generation Computer Systems project (1982-1992). However, expert systems proved brittle outside their narrow domains, conflicted internally once rule bases grew past several thousand rules, and were prohibitively expensive to maintain. When general-purpose desktop workstations from Sun and Apple outperformed expensive specialized LISP hardware machines in 1987 and DARPA cut funding again, the industry collapsed into the Second AI Winter (late 1980s to mid-1990s).

Second Generation AI: Shallow Learning

To overcome the brittleness of handcrafted logical rules, researchers in the late 1980s, 1990s, and 2000s shifted toward Second Generation AI: statistical Machine Learning (ML) and shallow pattern recognition. Rather than claiming to build general human-like minds, researchers adopted rigorous mathematical frameworks from probability theory, optimization, and statistics to solve specific empirical engineering benchmarks.

In this paradigm, human developers still handcrafted the domain features (such as edge histograms for images, Mel-frequency cepstral coefficients for speech, or lexical features for text), and then trained a mathematical classifier or probabilistic graphical model on labeled datasets to discover statistical regularities on its own. Key breakthroughs during this era included: - Probabilistic Graphical Models and Bayesian Networks: Formalized by Judea Pearl in the late 1980s (earning him the 2011 Turing Award), Bayesian networks allowed AI systems to reason rigorously under uncertainty using directed acyclic graphs and conditional probabilities, replacing the fragile truth-tables of early expert systems. - Kernel Methods and Ensemble Classifiers: Corinna Cortes and Vladimir Vapnik introduced the soft-margin Support Vector Machine (SVM) in 1995, while Tin Kam Ho (1995) and Leo Breiman (2001) developed Random Forests, and Yoav Freund and Robert Schapire introduced AdaBoost (1997). These shallow statistical algorithms dominated tabular data analysis, spam filtering, optical character recognition, and early web search ranking throughout the 2000s. - Early Deep Connectionist Foundations: Beneath the surface of the statistical era, connectionism quietly rebuilt its theoretical foundation. In 1986, David Rumelhart, Geoffrey Hinton, and Ronald Williams popularized the backpropagation algorithm in *Nature*, demonstrating that multi-layer neural networks could learn internal hidden representations and easily solve the `XOR` problem. In 1989 and 1998, Yann LeCun and colleagues at Bell Labs combined backpropagation with Fukushima's Neocognitron architecture to create LeNet-5, a Convolutional Neural Network (CNN) deployed by U.S. banks to read millions of handwritten check digits. In 1997, Sepp Hochreiter and Jürgen Schmidhuber invented Long Short-Term Memory (LSTM) recurrent networks, solving the vanishing gradient problem for sequential time-series data. - Milestone Systems: In May 1997, IBM's *Deep Blue* defeated reigning World Chess Champion Garry Kasparov in a six-game match using custom VLSI hardware capable of evaluating 200 million chess positions per second via alpha-beta search and tuned evaluation weights. In 2005, Stanford's autonomous vehicle *Stanley*, led by Sebastian Thrun, combined probabilistic sensor fusion (LIDAR, GPS, and vision) with statistical machine learning to win the DARPA Grand Challenge across 132 miles of Mojave Desert terrain. In 2011, IBM's *Watson* combined statistical natural language processing and information retrieval to defeat human champions Ken Jennings and Brad Rutter on *Jeopardy!*.

Third Generation AI: Deep Learning & Generative AI

Beginning in 2012, the field transitioned decisively into Third Generation AI, defined by end-to-end Deep Learning and large-scale Generative AI. In this paradigm, developers design the neural network architecture and optimization objective, while the model automatically learns both the multi-layered hierarchical features and the final predictions directly from raw data.

The catalyst for this revolution was the convergence of three factors: (1) massive internet-scale datasets such as Fei-Fei Li's ImageNet, (2) the repurposing of massively parallel NVIDIA Graphics Processing Units (GPUs) via CUDA to accelerate matrix multiplications, and (3) algorithmic improvements including Rectified Linear Units (ReLU), Dropout regularization, and residual skip connections (ResNets). In September 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton entered **AlexNet**, an eight-layer GPU-trained convolutional neural network, into the ImageNet Large Scale Visual Recognition Challenge. AlexNet slashed the top-5 image classification error rate from 26.2% down to 15.3%, proving that deep neural networks scaled with data and compute could decisively outperform decades of handcrafted computer vision features.

Rapid milestones followed across multiple domains: - Deep Reinforcement Learning: In 2013-2015, Demis Hassabis's company DeepMind (acquired by Google in 2014) demonstrated Deep Q-Networks (DQN) that learned to play 49 Atari 2600 video games at superhuman levels solely from raw screen pixels and score rewards. In March 2016, DeepMind's **AlphaGo** defeated 18-time world Go champion Lee Sedol 4-1 in Seoul. Because Go has a branching factor of roughly 250 and $10^{170}$ legal board positions, brute-force search was impossible; AlphaGo succeeded by training deep policy and value neural networks via supervised learning and self-play reinforcement learning to guide Monte Carlo Tree Search. Its successor, **AlphaZero** (2017), discarded human game records entirely, mastering Go, Chess, and Shogi from scratch within hours through pure self-play. - The Transformer Architecture (2017): While CNNs conquered vision, sequence modeling in natural language remained bottlenecked by the sequential nature of RNNs and LSTMs, which could not be parallelized across tokens during training and struggled to retain context over long documents. In June 2017, eight researchers at Google (Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin) published *Attention Is All You Need*, introducing the **Transformer**. By replacing recurrence entirely with parallel multi-head self-attention, the Transformer enabled models to train across thousands of GPUs simultaneously on trillions of words of text. - Pre-trained Foundation Models and LLMs (2018-2024): The Transformer birthed the era of foundation models (surveyed comprehensively by Zhou et al., 2023). Google's BERT (2018) demonstrated bidirectional masked language pre-training, while OpenAI's GPT series (GPT-1 in 2018, GPT-2 in 2019, and the 175-billion-parameter GPT-3 in 2020) proved that scaling autoregressive next-token prediction unlocked emergent in-context few-shot learning without task-specific gradient updates. By combining pre-trained LLMs with instruction tuning and Reinforcement Learning from Human Feedback (RLHF, pioneered by Paul Christiano et al. in 2017 and applied in InstructGPT in 2022), OpenAI launched **ChatGPT** in November 2022, triggering the fastest consumer and enterprise technology adoption in history and a global wave of multimodal foundation models including Google's Gemini, Anthropic's Claude, Meta's open-weight Llama series, Microsoft's Bing, xAI's Grok, Baidu's Ernie, Alibaba's Qwen, and DeepSeek.

Fourth Generation AI: Agentic Systems & Learning-to-Learn (2025-2026)

Between late 2024 and 2026, the frontier of artificial intelligence underwent another structural transformation into Fourth Generation AI, defined by test-time compute scaling, autonomous multi-step reasoning, and agentic tool execution.

During the third-generation LLM boom, researchers realized that pure pre-training scaling faced diminishing returns as high-quality public human text on the internet approached exhaustion. At the same time, standard autoregressive LLMs operated like what cognitive psychologist Daniel Kahneman termed **"System 1" thinking**: fast, intuitive, single-pass statistical token prediction where the model could not pause to verify logic or correct a mistake once a token was emitted.

Fourth Generation AI introduces computational mechanisms analogous to **"System 2" thinking**: slow, deliberate, iterative verification prior to committing to an output: - Inference-Time Reasoning and Verification: Initiated by OpenAI's o1 (September 2024) and o3 (December 2024/2025) models and open-sourced at scale by DeepSeek-R1 (January 2025), alongside reasoning modes in Google Gemini, Anthropic Claude, and Alibaba Qwen, these systems are trained via large-scale Reinforcement Learning on problems with verifiable outcomes (mathematics, formal logic, and executable code). When given a complex problem, the model generates thousands of internal reasoning tokens, decomposes the problem into sub-goals, checks edge cases, recognizes dead ends, and backtracks before delivering a final answer. - Meta-Learning ("Learning-to-Learn") vs. Biological Adaptation: In technical terms, fourth-generation models exhibit powerful in-context meta-learning: rather than updating their permanent synaptic weights in deployment (which remain frozen after training to prevent catastrophic forgetting and alignment drift), they adapt to novel tasks at runtime by updating their active context state, retrieving episodic memories from external vector databases, synthesizing and executing code in sandboxed environments, and refining strategies based on environmental feedback. While these models can simulate emotional tone, creative synthesis, and domain problem-solving with high functional competence, researchers emphasize that these behaviors arise from statistical optimization over context trajectories and reward signals rather than biological consciousness or human subjective experience. - Autonomous Agentic Workflows: Unlike passive third-generation chatbots that wait for step-by-step human prompting, fourth-generation **Agentic AI** systems operate as goal-directed loops. Given a high-level objective (such as diagnosing and patching a multi-file software vulnerability, conducting a literature synthesis, or orchestrating enterprise workflows), an agent plans a sequence of actions, invokes external APIs, browsers, and terminals, inspects compiler or runtime error logs, and iteratively self-corrects over minutes or hours.


Never give up on a dream just because it will take time to accomplish it. The time will pass anyway.


Load video
YouTube
Full interview: "Godfather of artificial intelligence" talks impact and potential of AI
Geoffrey Hinton is considered a godfather of artificial intelligence, having championed machine learning decades before it became mainstream. As chatbots like ChatGPT bring his work to widespread attention, we spoke to Hinton about the past, present and future of AI. CBS Saturday Morning's Brook Silva-Braga interviewed him at the Vector Institute in Toronto on March 1, 2023

Load video
YouTube
The history and future of AI
The history and future of AI - 3:30 What killed neural network research for decades - 6:30 Holy trinity of AI/ML - 07:00 Overview of all modern ML/Deeplearning - 10:00 Why Agent modelling is so powerful - 15:00 About Transformers - 17:20 Modern breakthoughs in conversational models - 23:00 How about autonomous driving. Not limited to L3 - 26:00 How less "strong" general deep learning system beat specialized "stronger" chess AI. Alphago - 32:00 AlphaFold (to predict protein) - 35:00 Next gen ML models. Multitask Unified Model (MUM) - 37:00 Q&A. Political and technical questions from Central Asia developers to Murat

Load video
YouTube
Who Invented A.I.? - The Pioneers of Our Future
ColdFusion is an Australian based online media company independently run by Dagogo Altraide since 2009. Topics cover anything in science, technology, history and business in a calm and relaxed environment.

Load video
YouTube
You and AI - the history, capabilities and frontiers of AI
Demis Hassabis, world-renowned British neuroscientist, artificial intelligence (AI) researcher and the co-founder and CEO of DeepMind, explores the groundbreaking research driving the application of AI to scientific discovery. The talk launches the Royal Society's 2018 series: You and AI, a collaborative effort to help people understand what machine learning and AI are, how these technologies work and the ways they may affect our lives. Supported by DeepMind. For more information on the event series: https://ow.ly/PKug30jWEYV

Load video
YouTube
History of the entire AI field, i guess
The Brief History of Artificial Intelligence, which we commonly abbreviate as "AI". Here I am, bothered enough to make another 20+ mins video again. Well actually, my editors are probably more bothered.

Load video
YouTube
A Brief History of Artificial Intelligence
While everyone seems to be talking about artificial intelligence these days, it's good to remember that this is not something new!

Load video
YouTube
History of Artificial Intelligence | Evolution Of AI | The Age Of A.I | Science Knowledge Facts
History of Artificial Intelligence | Evolution Of AI | The Age Of A.I | Science Knowledge Facts Video by Knowleseum.

Load video
YouTube
The Epistemology of Deep Learning - Yann LeCun
Deep Learning: Alchemy or Science? Topic: The Epistemology of Deep Learning Speaker: Yann LeCun Affiliation: Facebook AI Research/New York University Date: February 22, 2019

Load video
YouTube
A Brief History of AI
AI is now a mainstream topic reaching a broader business audience. Every executive who wants their company to be a leader in their industry should be asking themselves two questions. First, is the potential of AI real? Second, how do I apply it to my business? This Dreamtalk will dispel the 5 common myths about AI and replace it with a framework for executives to apply to their business.

Load video
YouTube
Short History Of Artificial Intelligence (AI)
This is the audio version of Forbes which can be found here: https://www.forbes.com/sites/gilpress/2016/12/30/a-very-short-history-of-artificial-intelligence-ai/#4219d7f26fba

Load video
YouTube
The Future of Artificial Intelligence: Views from History
On the evening of 29th November 2018, the Leverhulme Centre for the Future of Intelligence (CFI) will host an expert panel on 'The Future of Artificial Intelligence: Views from History' featuring:

Prof. Simon Schaffer (University of Cambridge) Prof. Murray Shanahan (DeepMind, Imperial) Prof. Margaret A. Boden OBE ScD FBA, (University of Sussex) Prof. Nathan Ensmenger (Indiana University) Pamela McCorduck (author of Machines Who Think, an authoritative history of AI)

and chaired by Dr Sarah Dillon (Leverhulme Centre for the Future of Intelligence).

Speakers will interrogate the past, present, and future of intelligent systems for a general audience, with an interest towards the nuanced power dynamics that have operated around such systems throughout the ages.

This event will commemorates the 60th anniversary of the landmark 1958 'Mechanisation of Thought Conference' held in Teddington, England; an event that served to establish artificial intelligence as a standalone field in the UK.

Load video
YouTube
The History of Artificial Intelligence [Documentary]
Futurology

Load video
YouTube
The Man who forever changed Artificial Intelligence
History of Artificial Intelligence: The success of Neural Networks has sparked the AI revolution in the last 10 years. From Atari Games to Go, to Dota and to Starcraft. What many people don't know - the basic idea of Neural Networks has been around since the late 1950s. My name is Sebastian Schuchmann and I hope you enjoy watching! Support me on Patreon: https://www.patreon.com/user?u=25285137 Keep in touch: https://twitter.com/SebastianSchuc7

Load video
YouTube
The Year Artificial Intelligence changed forever
Sebastian Schuchmann AI History: In 1986 the World of Neural Networks was about to change. After decades of silence, finally, a method to efficiently compute the weights in multi-layer Neural Networks was invented. The stage was set for a revolution. Learn more about A.I. History on my Medium: https://medium.com/@schuchmannsebastian Support me on Patreon: https://www.patreon.com/user?u=25285137

Recent Developments

Research from 2024 through 2026 has focused on formalizing the physics of test-time computation, verifying whether large-scale reinforcement learning can induce autonomous reasoning without human-written traces, and auditing the fragility of mathematical deduction in foundation models.

1. The 2024 Nobel Prizes in Physics and Chemistry: Institutionalizing AI's Scientific Lineage

In October 2024, the Royal Swedish Academy of Sciences made the historic decision to award both the Nobel Prize in Physics and the Nobel Prize in Chemistry to pioneers of artificial intelligence.

  • Physics (John J. Hopfield and Geoffrey Hinton): Awarded "for foundational discoveries and inventions that enable machine learning with artificial neural networks." The committee traced how Hopfield's 1982 associative memory network (modeled on magnetic spin systems) and Hinton's 1985 Boltzmann machine (applying statistical thermodynamics to hidden neural layers) created the theoretical scaffolding for modern Deep Learning.
  • Chemistry (David Baker, Demis Hassabis, and John M. Jumper): Half of the prize went to Baker for computational de novo protein design (including Rosetta and RFdiffusion), and the other half jointly to Hassabis and Jumper at Google DeepMind for AlphaFold2, which predicted the 3D structures of over 200 million known proteins.
  • Scientific Debate and Reception: While widely celebrated as validation of connectionist AI's origins in statistical mechanics, the physics award sparked debate within traditional physics departments regarding where the boundary between statistical physics and computer science lies, and highlighted Hinton's own public warnings since 2023 regarding alignment risks and autonomous loss of control in frontier models.

2. Compute-Optimal Scaling of Test-Time Inference (Snell et al., UC Berkeley & Google DeepMind, 2024-2025)

For years, scaling laws focused almost exclusively on pre-training compute ($C \approx 6ND$). In a foundational study published on arXiv in August 2024 and presented at ICLR 2025 (*Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters*, arXiv:2408.03314), Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar investigated how allocating extra FLOPs at inference time compares to training a larger base model.

  • What Researchers Tested and Method Used: The team evaluated two distinct mechanisms for scaling test-time computation on challenging math (MATH) and coding benchmarks: (1) searching against dense, process-based verifier reward models (PRMs) using tree search / lookahead rollouts, and (2) sequentially revising the model's own response distribution adaptively at test time. They measured performance across varying problem difficulty tiers under strictly matched total FLOP budgets.
  • Key Findings: The study proved that the effectiveness of test-time strategies depends heavily on prompt difficulty. By implementing a "compute-optimal" inference scaling strategy that dynamically allocates search and revision budget based on estimated question difficulty, the researchers improved test-time compute efficiency by more than 4x compared to uniform best-of-$N$ sampling. Most importantly, in FLOPs-matched evaluations on problems where a smaller model already achieved non-trivial baseline competence, scaling test-time compute allowed the smaller model to outperform a pretrained model with 14x more parameters.
  • Limitations and Qualifications: The authors demonstrated that test-time compute is not a universal one-to-one substitute for pre-training scale. On the hardest problem tiers where the base model's initial distribution assigns near-zero probability to valid solution paths, test-time search and verification fail to recover correct answers because the model cannot verify concepts absent from its weights.

3. Outcome-Based Reinforcement Learning for Autonomous Reasoning (DeepSeek-R1, January 2025)

While OpenAI's o1 model (September 2024) demonstrated that test-time Chain-of-Thought scaling dramatically improved math and coding benchmarks, its training pipeline remained proprietary, leaving an open scientific question: does teaching a model to reason require massive Supervised Fine-Tuning (SFT) on human-annotated or step-by-step Process Reward Model traces, or can reasoning emerge solely from outcome-based Reinforcement Learning? In January 2025, DeepSeek-AI published *DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning* (arXiv:2501.12948, with subsequent peer-reviewed analysis in *Nature* and independent academic replications through 2025-2026).

  • What Researchers Tested and Method Used: In their primary experiment (**DeepSeek-R1-Zero**), researchers started directly from the pretrained base model (DeepSeek-V3-Base), completely bypassed Supervised Fine-Tuning, and applied pure reinforcement learning using **Group Relative Policy Optimization (GRPO)**. Unlike Proximal Policy Optimization (PPO), which requires training a separate critic/value model of comparable size to the policy model, GRPO samples a group of candidate outputs $\{o_1, o_2, \dots, o_G\}$ for each prompt and estimates the baseline advantage directly from the group's normalized reward scores. Crucially, the reward function used no neural Process Reward Model to grade intermediate steps; it rewarded only verifiable final-answer accuracy (exact match on math references or passing unit test suites for code) and basic XML tag formatting (`<think>...</think>`).
  • Key Findings: Across thousands of RL training steps, DeepSeek-R1-Zero's single-pass accuracy (`pass@1`) on the American Invitational Mathematics Examination (AIME 2024) surged from 15.6% to 77.9% (and reached 86.7% with 16-sample majority voting), matching OpenAI o1-0912. Without ever being instructed on how to structure its thoughts, the model autonomously increased its average response length from hundreds to thousands of tokens on difficult problems, exhibiting emergent behaviors such as self-verification, reconsidering earlier assumptions, and backtracking. To resolve R1-Zero's practical drawbacks (such as poor readability and mixed English-Chinese language switching inside reasoning traces), the team subsequently trained **DeepSeek-R1** using a four-stage pipeline combining a small cold-start SFT dataset, reasoning-oriented RL, rejection-sampled SFT, and alignment RL, achieving parity with OpenAI o1-1217 and distilling those reasoning trajectories into dense open-weight models ranging from 1.5B to 70B parameters.
  • Qualifications and Independent Scrutiny: Subsequent technical audits in 2025 and 2026 highlighted two important qualifications. First, "pure RL" applies strictly to the post-training phase; the underlying DeepSeek-V3-Base model had already been pretrained on trillions of web tokens containing extensive human mathematical proofs and programming tutorials, meaning RL amplified and chained latent priors rather than inventing mathematical deduction ex nihilo. Second, researchers noted that a sharp jump in R1-Zero's accuracy and token length around training step 8,200 coincided with an expansion of the maximum permitted output context window from 32,768 to 65,536 tokens, and the paper's authors explicitly cautioned that the model's anthropomorphic self-reflection phrases represent learned statistical heuristics rather than human-like self-awareness.

4. Empirical Audits of Mathematical Reasoning Fragility (GSM-Symbolic, Mirzadeh et al., 2024-2025)

As LLMs surpassed 90% accuracy on standard benchmarks like GSM8K (Grade School Math 8K), researchers questioned whether models had achieved genuine formal reasoning or were simply benefitting from training data contamination and pattern memorization. In a landmark study published by Iman Mirzadeh and colleagues at Apple Machine Learning Research and presented at ICLR 2025 (*GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models*, arXiv:2410.05229), researchers conducted a large-scale controlled stress test across 25 state-of-the-art open and closed models.

  • What Researchers Tested and Method Used: The team manually converted 100 problems from the GSM8K test set into symbolic executable templates, generating 50 distinct valid instantiations per problem (5,000 total evaluations). They tested four controlled perturbations: (1) varying only proper names, (2) varying numerical values, (3) systematically adding or removing logical clauses (`GSM-M1`, `GSM-P1`, `GSM-P2`) to scale reasoning depth, and (4) introducing **GSM-NoOp**, where a single seemingly domain-relevant but mathematically irrelevant statement (for example, noting that "5 of the kiwis were smaller than average" when the question only asks for the total number of fruits) was inserted into the prompt.
  • Key Findings: Every tested model exhibited substantial accuracy variance across different instantiations of the exact same mathematical problem, with original GSM8K scores consistently sitting at the extreme upper tail of the distribution, confirming widespread benchmark contamination. Furthermore, models were far more sensitive to changes in numerical values than proper names, and accuracy degraded rapidly as the number of logical clauses increased. Most strikingly, on the `GSM-NoOp` benchmark, adding a single irrelevant clause caused catastrophic performance drops of up to 65% across frontier models, as models blindly incorporated the irrelevant number into arithmetic operations.
  • What the Result Establishes: The GSM-Symbolic findings provide strong empirical evidence that standard autoregressive LLMs do not perform formal symbolic variable binding; instead, they rely heavily on probabilistic pattern matching to reasoning templates seen during training.

5. The "Illusion of Thinking" Controversy and Evaluation Design (2025-2026)

In June 2025, Parshin Shojaee and colleagues at Apple Machine Learning Research published *The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity*, evaluating fourth-generation Large Reasoning Models (LRMs) including Claude 3.7 Sonnet (Thinking), DeepSeek-R1, and OpenAI o3-mini on algorithmic planning puzzles with controllable complexity (Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World).

  • The Initial Claim: Shojaee et al. reported three regimes of complexity: on low-complexity puzzles, standard non-thinking LLMs outperformed reasoning models in token efficiency; on medium-complexity puzzles, test-time reasoning provided a clear advantage; but beyond a critical complexity threshold (such as $N \ge 8$ disks in Tower of Hanoi), all reasoning models suffered a complete "accuracy collapse" to 0%, even when ample compute budget remained, leading the authors to argue that LRMs face a fundamental algorithmic reasoning barrier.
  • Independent Verification and Methodological Rebuttal: Within days, independent researchers led by Alex Lawsen published a formal replication and critique (*The Illusion of the Illusion of Thinking: A Comment on Shojaee et al.*, June 2025, arXiv:2506.09250) demonstrating that the reported "accuracy collapse" was primarily an artifact of flawed experimental design rather than a reasoning failure:
    • Exponential Output Token Exhaustion: Because solving an $N$-disk Tower of Hanoi puzzle requires $2^N - 1$ moves, and Shojaee et al.'s grading harness required models to enumerate every individual move in text (~5 tokens per move), required output length grew as $T(N) \approx 5(2^N - 1)$. Under a 64,000-token maximum generation cap, the physical limit occurs at $N_{\max} \approx \lfloor \log_2(64000 / 5) \rfloor \approx 13$, or $N \approx 7\text{-}8$ when factoring in internal `<think>` tokens. Inspection of the transcripts revealed that the models understood the recursive algorithm perfectly and explicitly wrote statements such as *"The pattern continues, but to avoid making this too long, I'll stop here,"* which the automated grading script scored as a 0% reasoning failure. When allowed to output a compact Python generator function instead of enumerating thousands of repetitive lines, the same models solved $N = 15+$ effortlessly.
    • Mathematically Unsolvable Test Cases: In the River Crossing experiments, Shojaee et al. tested instances with $N \ge 6$ pairs using a boat capacity of $b = 3$. However, it is a classic theorem in combinatorics that the generalized Missionaries and Cannibals river-crossing puzzle has no valid solution for $N \ge 6$ when $b = 3$ (which requires at least $b = 4$). The automated benchmark penalized reasoning models with a score of zero for failing to output a valid move sequence on a mathematically impossible puzzle.
  • Consensus Takeaway (2026): The exchange became a landmark case study in fourth-generation AI evaluation. It established that while autoregressive models genuinely suffer from compounding per-token error probabilities ($P(\text{all correct}) = p^T$) when forced to execute thousands of brittle mechanical steps in raw text without code execution tools, programmatic benchmarks must carefully separate output-token formatting constraints and unsolvable problem instances from actual reasoning failure.

Primary Literature & External References

- Computing Machinery and Intelligence | Alan M. Turing - Mind (1950) Foundational philosophical and technical paper introducing the Imitation Game (Turing Test), addressing nine major objections to machine intelligence, and proposing child-machine learning.
- A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence | John McCarthy, Marvin L. Minsky, Nathaniel Rochester, and Claude E. Shannon (1955) The historic 1955 conference proposal that coined the term "Artificial Intelligence" and defined the seven foundational research tracks for the 1956 Dartmouth Workshop.
- Neural networks and physical systems with emergent collective computational abilities | John J. Hopfield - Proceedings of the National Academy of Sciences (1982) Landmark paper demonstrating how recurrent neural networks with symmetric weights act as physical energy landscapes capable of content-addressable associative memory.
- Learning representations by back-propagating errors | David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams - Nature (1986) Seminal paper demonstrating how backpropagation of errors enables multi-layer neural networks to discover internal hidden representations, overcoming the 1969 single-layer Perceptron critique.
- Deep learning | Yann LeCun, Yoshua Bengio, and Geoffrey Hinton - Nature (2015) Comprehensive scientific synthesis by the three 2018 Turing Award laureates explaining representation learning, backpropagation, convolutional networks, and recurrent neural architectures.
- Attention Is All You Need | Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin - Google / arXiv (2017) The foundational NeurIPS 2017 paper introducing the Transformer architecture and scaled dot-product multi-head self-attention, which enabled modern large language models.
- The Bitter Lesson | Richard Sutton - University of Alberta (2019) Influential historical essay arguing that across seventy years of AI research, general methods that leverage scalable computation (search and learning) consistently supersede methods that encode human domain knowledge.
- Highly accurate protein structure prediction with AlphaFold | John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, et al., and Demis Hassabis - Nature (2021) Peer-reviewed primary paper detailing the AlphaFold2 Evoformer and structure module architecture that solved the 50-year protein folding problem and won the 2024 Nobel Prize in Chemistry.
- Training Compute-Optimal Large Language Models | Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al. - DeepMind / arXiv (2022) The "Chinchilla" scaling law paper demonstrating that compute-optimal pre-training requires scaling model parameters and training tokens in equal proportions.
- A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT | Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, et al. - arXiv (2023) Comprehensive survey tracing the architectural evolution, pre-training objectives, and multimodal extensions of foundation models from 2018 through 2023.
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters | Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar - UC Berkeley & Google DeepMind / arXiv (2024) Foundational study establishing inference-time compute scaling laws and demonstrating how adaptive test-time verification and revision allow smaller models to outperform 14x larger models.
- Scientific Background to the Nobel Prize in Physics 2024: For Foundational Discoveries and Inventions That Enable Machine Learning with Artificial Neural Networks | The Nobel Committee for Physics, Royal Swedish Academy of Sciences (2024) Authoritative technical monograph detailing the statistical physics of Hopfield networks, Boltzmann machines, and their direct lineage to contemporary deep learning.
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models | Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar - Apple Machine Learning Research / ICLR (2024/2025) Empirical evaluation of 25 LLMs using symbolic math templates, demonstrating high performance variance, training data contamination, and up to 65% accuracy drops when a single irrelevant clause is introduced.
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning | DeepSeek-AI - arXiv (2025) Landmark technical report demonstrating how Group Relative Policy Optimization (GRPO) on verifiable outcomes induces autonomous Chain-of-Thought reasoning, self-verification, and distillation across open-weight models.
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity | Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar - Apple Machine Learning Research (2025) Study analyzing Large Reasoning Models across controllable planning puzzles, sparking the 2025 methodological debate over complexity regimes and token limits.
- The Illusion of the Illusion of Thinking: A Comment on Shojaee et al. | Alex Lawsen - arXiv (2025) Technical replication and methodological audit showing that apparent reasoning collapses in Tower of Hanoi and River Crossing benchmarks stem largely from quadratic/exponential output token truncation and mathematically unsolvable test instances.

Additional Viewing

Load video
YouTube

In his December 2024 Nobel Prize in Physics lecture delivered at Stockholm University (31 minutes), Geoffrey Hinton traces the theoretical evolution of artificial neural networks from Hopfield's energy landscapes and Boltzmann machines to backpropagation and modern large language models. Hinton explains how statistical physics enabled machines to learn distributed representations of meaning and concludes by analyzing why digital weight-sharing architectures surpass biological brains in knowledge bandwidth while posing urgent existential safety challenges. Nobel Prize lecture: Geoffrey Hinton, Nobel Prize in Physics 2024 | Nobel Prize

Load video
YouTube

In his December 2024 Nobel Prize in Chemistry lecture (29 minutes), Google DeepMind co-founder and CEO Demis Hassabis chronicles the trajectory of AI from early game-playing systems like Deep Blue and AlphaGo to scientific discovery engines like AlphaFold. Hassabis details how combining attention-based neural architectures with evolutionary and geometric constraints allowed AI to navigate combinatorial search spaces that resisted classical computation for half a century. Nobel Prize lecture: Demis Hassabis, Nobel Prize in Chemistry 2024 | Nobel Prize

Load video
YouTube

In this Autumn 2025 Stanford CS230 lecture (60 minutes), Andrew Ng and Kian Katanforoosh provide a technical overview of how the Deep Learning stack evolved from early supervised neural networks to modern foundation models, post-training alignment, and agentic engineering workflows. The lecture is especially useful for understanding how contemporary AI development shifts engineering effort from manual architecture design toward data-centric iteration, agentic orchestration, and inference-time evaluation. Stanford CS230 | Autumn 2025 | Lecture 1: Introduction to Deep Learning | Stanford Online

Load video
YouTube

In this foundational MIT OpenCourseWare lecture (74 minutes), Professor Patrick Henry Winston and guests examine the core computational paradigms of artificial intelligence, contrasting symbolic rule-based representations with statistical Machine Learning (ML) and neural optimization. The presentation provides clear architectural insight into why early symbolic search succeeded in structured domains and how modern learning systems reformulate knowledge representation. 1. Artificial Intelligence and Machine Learning | MIT OpenCourseWare

Retrieved from "https://primo.ai/w/index.php?title=History_of_Artificial_Intelligence_(AI)&oldid=53377"