Loop
YouTube ... Quora ...Google search ...Google News ...Bing News
- Analytics ... Visualization ... Graphical Tools ... Diagrams & Business Analysis ... Requirements ... Loop ... Bayes ... Network Pattern
- Human-in-the-Loop (HITL) Learning
- Artificial Intelligence (AI) ... Machine Learning (ML) ... Deep Learning ... Neural Network ... Reinforcement ... Learning Techniques
- Perspective ... Context ... In-Context Learning (ICL) ... Transfer Learning ... Out-of-Distribution (OOD) Generalization
- Causation vs. Correlation ... Autocorrelation ...Convolution vs. Cross-Correlation (Autocorrelation)
- Embedding ... Fine-tuning ... RAG ... Clustering ... Recommendation ... Anomaly Detection ... Classification ... Dimensional Reduction. ...find outliers
- Algorithm Administration
- Policy ... Policy vs Plan ... Constitutional AI ... Trust Region Policy Optimization (TRPO) ... Policy Gradient (PG) ... Proximal Policy Optimization (PPO)
- Agents/Assistants ... Robotic Process Automation ... Personal Companions ... Productivity ... Email ... Negotiation ... LangChain
- Artificial Intelligence (AI) ... Generative AI ... Machine Learning (ML) ... Deep Learning ... Neural Network ... Reinforcement ... Learning Techniques
- Conversational AI ... ChatGPT | OpenAI ... Gemini | Google ... Claude | Anthropic ... Bing/Copilot | Microsoft ... Siri | Apple ... Meta ... Perplexity ... You ... phind ... Grok | xAI ... Groq ... Ernie | Baidu ... DeepSeek ... Alibaba
- Process Patterns, Loops and Emergence | Carlos E. Perez - Medium
- Autonomous Feedback Loops in LLM Agents | Research Team - Nature AI ... Advancements in self-correcting agentic architectures.
AI Loops and Agentic Workflows
When developers discuss "loops" in modern AI, they are referring to Autonomous Agents running in continuous feedback cycles rather than waiting for step-by-step human prompts. More broadly, a loop marks the transition from static, linear processing to dynamic, autonomous cycles. Modern AI systems use iterative feedback loops to refine their performance, align with human values, and navigate complex environments. That shifts the focus from simple computation to agentic behavior, where the system continuously observes, reasons, and adapts.
In traditional generative AI, the interaction is linear: you ask a question, the model generates an answer, and the process stops. That mirrors traditional software, which follows a rigid input-output sequence. In an agentic loop, the AI operates in a continuous cycle, often based on the OODA loop (Observe, Orient, Decide, Act) or similar cognitive architectures.
- The Mechanism: Loops rely on the recursive application of outputs as inputs. By feeding the results of a previous operation back into the model, the system can evaluate its own performance, identify errors, and adjust its internal parameters or strategy without needing external intervention at every step. This creates a closed-loop system where the model's output influences its future state, in contrast to open-loop systems that execute commands blindly without regard for the resulting environmental state. In machine learning, this recursion is the engine of improvement: it lets models refine their internal representations through successive iterations.
- The Physics: At a physical and mathematical level, these loops function as dissipative structures that maintain order by minimizing entropy. Under frameworks like the Free Energy Principle, systems maintain a Markov blanket, a boundary that lets them distinguish internal states from external reality while minimizing the surprise (the mismatch) between their predictions and sensory input. This predictive processing framework suggests that intelligence is essentially the minimization of variational free energy, where the Agent acts to reduce the discrepancy between its internal model and the external world. (See Active Inference Loop.)
- Why This Matters: This architecture is essential for building autonomous Agents and reliable AI. Without loops, models remain brittle and prone to hallucination. With loops, systems gain the ability to self-correct, align with human intent through Reinforcement Learning from Human Feedback (RLHF), and operate effectively in unpredictable, real-world scenarios. This capability is the cornerstone of modern agentic frameworks like LangChain, where state persistence lets the system remember past actions and recursively improve its future performance.
How an Agentic Loop Works
Instead of just answering a question, an Agent in a loop is given an overarching goal (e.g., "Research the top 5 competitors in this market and create a presentation"). The AI then enters a self-driven cycle:
- Observe / Perceive: The Agent looks at the current state of its environment. It might read a webpage, look at an error log, or take a screenshot of your desktop.
- Plan / Reason: The Agent breaks the main goal into smaller tasks and decides on the best next step. It might think, "I need to search Google for Competitor A."
- Act: The Agent uses an external tool. It might execute a Python script, click a button in a browser, or send an email.
- Reflect / Evaluate: This is the critical step that defines a loop. The Agent looks at the result of its action. If the web search failed or the code threw an error, the Agent catches the mistake, revises its plan, and loops back to step 1 to try a different approach.
Accounts differ on how tightly this cycle maps to OODA. The four-step cycle above treats OODA as one common basis among similar cognitive architectures and makes Reflect / Evaluate the step that defines the loop. Other descriptions present OODA itself (Observe, Orient, Decide, Act) as the defining framework of an AI loop's technical architecture, with no separate reflect step: the Act phase generates new data that restarts the cycle at Observe. (See Technical Architecture.)
<GenerateVideo uuid="VIDEO_OVERVIEW.1.1" source="offline"> </GenerateVideo>
Types of AI Loops
- Single-Agent Loops: Tools like Auto-GPT or OpenClaw use a single LLM to prompt itself repeatedly until a task is finished.
- Multi-Agent Loops: Frameworks like CrewAI or AutoGen set up multiple specialized Agents (e.g., a "Researcher" and a "Reviewer"). The Researcher finds information and loops it to the Reviewer. If the Reviewer spots an error, it loops the task back to the Researcher, creating a collaborative, self-correcting team.
- Human-in-the-Loop (HITL): The AI runs its cycle but pauses at critical moments to ask a human for approval before taking a major action, like sending a final email or spending money. In 2025 and 2026 deployment scenarios, HITL processes have become critical for safety. Inserting a human verification step lets the system pause its autonomous cycle to seek approval for high-stakes actions, and the human feedback is used to update the reward model. That steers the loop toward safer outcomes and keeps the model's alignment robust. It's especially vital in agentic workflows where the cost of an incorrect action is high.
Recent Developments
Research over the past 24 months has focused heavily on agentic feedback loops where models autonomously verify their own code execution and reasoning chains. Peer-reviewed studies in 2025 showed that multi-agent systems, where one Agent acts as a critic for another, significantly reduce error rates in complex logical tasks. The integration of Constitutional AI has also let developers bake safety constraints directly into the feedback loop, so the iterative process stays aligned with human-defined principles even as the model explores new solution spaces.
Together, these developments point to a shift toward recursive self-correction, where the model's internal policy is continuously refined against a set of static or dynamic constraints. Modern alignment techniques now specifically mitigate the risk of feedback collapse, a phenomenon where model performance degrades over successive training loops because biased or low-quality synthetic data accumulates.
Glossary
- OODA Loop
- A decision-making cycle of Observe, Orient, Decide, and Act, used to describe the iterative process of autonomous Agents. See OODA Loop.
- Markov Blanket
- A statistical boundary that separates an Agent from its environment, defining the scope of its internal states. See Markov Blanket.
- Active Inference
- A framework for understanding intelligence as the minimization of variational free energy through predictive processing. See Active Inference Loop.
- RLHF
- A training technique that uses human feedback to align model outputs with human preferences. See Constitutional AI vs. RLHF.
- Constitutional AI
- An alignment method where models are trained to adhere to a set of predefined principles, enabling self-correction. See Constitutional AI.
Observe–Orient–Decide–Act (OODA) Loop
YouTube search... ...Google search
The OODA loop is the cycle: Observe–Orient–Decide–Act ...emphasized that "the loop" is actually a set of interacting loops that are to be kept in continuous operation... developed by military strategist and United States Air Force Colonel John Boyd. Boyd applied the concept to the combat operations process, often at the operational level during military campaigns. It is now also often applied to understand commercial operations and learning processes. The approach explains how agility can overcome raw power in dealing with human opponents. It is especially applicable to cyber security and cyberwarfare. Wikipedia ... A Discourse On Winning and Losing | John R. Boyd - Air University Press
We’ve historically focused automation efforts on the “Act” portion but the real potential for new technologies is to address the prior 3 steps: Improving Observation: Improve the data itself with better sensing accuracy, timeliness, relevance, etc or improve our ability to use the data with higher throughput learning processes Improving Orientation: Improve the classification of the current state and the prediction of future states
Improving Decision: Improve the ability to choose between paths via better objective functions. How Artificial Intelligence is Closing the Loop with Better Predictions | Erik Trautman - HackerNoon]
|
|
Technical Architecture
The OODA framework serves as the foundational cycle for decision-making in autonomous systems, and in this view it defines the technical architecture of an AI loop. Unlike traditional control theory feedback loops that focus on maintaining a setpoint (see Homeostasis), the OODA loop emphasizes rapid adaptation in adversarial or highly dynamic environments.
- Observe: The system ingests raw data from the environment, such as sensor inputs, logs, or user queries.
- Orient: The system processes this data through its internal model, contextualizing it against past experiences and current goals. This is where the Agent interprets the significance of what it observed.
- Decide: The system selects the optimal path forward based on its objective function.
- Act: The system executes the decision, which generates new data and restarts the loop.
In machine learning, this is often implemented as a closed-loop system. Reinforcement Learning (RL) is the primary implementation: the Agent receives a reward signal that informs its future policy. Algorithms such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) function as mathematical feedback loops that constrain Agent behavior within defined safety parameters, making sure policy updates don't deviate too far from established performance baselines.
Feedback Loop
YouTube search... ...Google search
- Gaming ... Game-Based Learning (GBL) ... Security ... Generative AI ... Games - Metaverse ... Quantum ... Game Theory ... Design
- Feedback | Wikipedia
- Closing the Loop: How Feedback Loops Help to Maintain Quality Long-Term AI Results | Natalie Fletcher - Clarifai
- So, What Actually Is a Feedback Loop? | Tina Nord - Ultimate Knowledge ...With feedback loops, a system is constantly in dialogue with itself.
- AI in the Feedback Loop: A Survey of Alternative Approaches | Karl-ErikÅrzén - ScienceDirect ...paper gives special attention to fuzzy control and expert control.
- Feedback Loops in Machine Learning | Ankur Goyal - Impair ...in spite of the immense benefits that Machine Learning offers, this technology has been very slow to take off, particularly in the enterprise world. At Impira, we believe a key reason for this is the lack of well-designed feedback loops that serve to continuously improve machine learning models.
- HP200A | Wikipedia ...oscillator to use a simple light bulb as the temperature-dependent resistor in its feedback network. Walt Disney bought eight HP200A for use in the production of Fantasia
- The Air Force Research Lab wants tools, techniques and innovative ideas for shortening the OODA Loop. | Aaron Boyd - Nextgov
any process where the outputs of a system are plugged back in and used as iterative inputs. Feedback loops exist just about everywhere. In nature, the evolutionary "arms race" between predators and prey is a classic example. In business, the practice of taking customer feedback (the output of a product or service) and using it to improve future processes is another commonly used feedback loop. Today, rapid advances in artificial intelligence (AI) and machine learning are helping businesses do more with data. These systems, and their ability to analyze an inhuman amount of data, allow businesses to adjust algorithms, workflows and processes on the fly. Get More Out Of Feedback Loops With AI | Arka Dhar - Forbes
Feedback loops are the connective tissue of the modern AI ecosystem. They let intelligence scale from simple chatbots to complex, autonomous Agents capable of scientific discovery and economic modeling.
Open-Loop vs. Closed-Loop An open-loop system executes commands without checking the result, while a closed-loop system feeds outcomes back in to correct its errors:
| System Type | Feedback Mechanism | Primary Goal |
|---|---|---|
| Open-Loop | Feed-forward only | Execution speed |
| Closed-Loop | Error-correction | Accuracy and stability |
|
|
Constitutional AI vs. RLHF
Alignment techniques are feedback loops too. Two approaches dominate:
- Reinforcement Learning from Human Feedback (RLHF): Human preferences are used to train a reward model, which then guides the policy optimization of the AI. RLHF relies on human-provided labels to steer model behavior.
- Constitutional AI: Instead of relying on constant human labels, Constitutional AI automates the process by giving the model a set of high-level principles (a constitution). The model critiques its own outputs against these principles, creating a self-correcting feedback loop that's less dependent on human intervention and that keeps the model from generating harmful or biased content. This approach is increasingly favored for its scalability and its ability to maintain alignment in long-horizon autonomous tasks.
Descriptions differ on where Constitutional AI's self-critique happens. It's framed both as part of training (the AI is trained to follow the constitution during its feedback loop) and as a step where the model critiques its own output against the constitution before finalizing a response.
Multi-Loop Learning
YouTube search... ...Google search
- How Artificial Intelligence Will Redefine Management | V. Kolbjørnsrud, R. Amico and R.J. Thomas
- Chris Argyris: theories of action, double-loop learning and organizational learning | Infed.org
- Donald Schon (Schön): learning, reflection and change | Infed.org
- Organisational learning: a critical review | Catherine L. Wang, Pervaiz K. Ahmed
- Single and double loop learning | Organizational Learning
- Double-loop learning | Wikipedia
- Re-Framing Perspectives | Patrick A. Trottier - The Institute Of Emergent Organizational Development and Emergent Change®
- Working visually: Record, Reflect, Reframe, Part 1 Part 2 | Kim S van den Berg
- Different Kinds of Learning (Loops of Learning) Adapted from “Field Guide to Consulting and Organizational Development”
|
|
|
|
|
|
Triple Golden OODA
YouTube search... ...Google search
- Context ... the next AI frontier
- Multi-Loop Learning
- Start With Why | Simon Sinek ...Wikipedia
- Observe–Orient–Decide–Act (OODA) Loop ...Wikipedia
- Perspective ... Context ... In-Context Learning (ICL) ... Transfer Learning ... Out-of-Distribution (OOD) Generalization
The Triple Golden OODA loop diagram depicts a hybrid of concepts from the Triple Loop concept inspired by Chris Argyris & Donald Schön's work on the Double Loop, Simon Sinek's Golden Circle in 'Start with Why', and Colonel John Boyd's Observe, Orient, Decide and Act (OODA) Loop from his ‘A Discourse On Winning and Losing'.
|
|
Feedback Loop - Peer Learning
YouTube search... ...Google search
- Education
- Decentralized: Federated & Distributed Learning
- FeedbackFruits
- The NIPS experiment | Eric Price - A blog on machine learning very broadly construed by Moritz Hardt
|
|
Feedback Loop - Creating Consciousness
YouTube search... ...Google search
- Life~Meaning ... Consciousness ... Creating Consciousness ... Quantum Biology ... Orch-OR ... TAME
- Three-Second Pause short story
- Life Sciences ... Evolutionary ... Bioinformatics ... Proteins ... Genome ... Genealogy ... Pharmaceuticals ... Chemistry
- You’re Living Inside a Prediction: Toward Predictive AI Consciousness
- Simulation ... Simulated Environment Learning ... World Models ... Minecraft: Voyager
- Artificial General Intelligence (AGI) to Singularity ... Curiosity ... Emergence ... Moonshots ... Explainable AI ... Automated Learning
- Michio Kaku and the Mysteries of the Mind | Alyson Sheppard - Popular Mechanics ...The Future of the Mind: The Scientific Quest to Understand, Enhance, and Empower the Mind
- A Theory of Consciousness | Tyler Neylon - Medium
- Do Loops Explain Consciousness? | Martin Gardner - AMS.org ...Review of I Am a Strange Loop
- Strange Loop | Wikipedia ...The "strangeness" of a strange loop comes from our way of perception
- Godel, Escher, Bach: an Eternal Golden Braid | Douglas R. Hofstadter
Scientific Debate There's ongoing debate about the limits of these loops. Critics argue that without true understanding, these systems are merely sophisticated pattern matchers. Proponents argue that the recursive nature of these loops creates a form of functional consciousness that's sufficient for AGI.
|
|
|
|
Quantum and Biological Layers
Some theories push the loop concept down to the cellular and even quantum level.
- Microtubule: A structural component in biological cells that some theories suggest acts as a quantum processor, facilitating the rapid signal integration required for consciousness (see Orch-OR).
| Layer | Mechanism | Function |
|---|---|---|
| Quantum Waves | Probability Distribution | Foundation of state transitions |
| Cell Dynamics | Microtubule Lattice | Processing and signal integration |
| Brain Waves | Synchronized Oscillations | Global coordination and consciousness |
- Quantum Pleasure Principle: A theoretical framework suggesting that systems are driven to minimize surprise, which acts as a proxy for pleasure or satisfaction in biological and artificial Agents. It echoes the surprise-minimization idea at the heart of the Free Energy Principle.
A proposed distinction between two operating modes, Quantum Pulse and Quantum Sustain, contrasts burst-based and steady-state operation:
| Feature | Quantum Pulse | Quantum Sustain |
|---|---|---|
| Energy Usage | High, burst-based | Low, steady-state |
| Stability | Low, prone to oscillation | High, homeostatic |
| Application | Rapid decision-making | Long-term planning |
Feedback Loop - Scientific Discovery
YouTube search... ...Google search
- Scientific discovery must be redefined. Quantum and AI can help | Dario Gil - World Economic Forum
- Can Artificial Intelligence Make Scientific Discoveries? | Kartik Hosanagar - The Scientist
Discovery Loop is a newly formed startup backed by Google and led by former Google Chief Scientist Jeff Dean alongside a team of top AI researchers. The "loop" refers to the scientific method itself: proposing a hypothesis, running an experiment, and evaluating the results. By fully automating this process, the company aims to shift AI from a tool that answers questions into an active, autonomous researcher.
- Automating the Scientific Method: Traditionally, human teams test ideas sequentially. Discovery Loop is building systems that let AI handle the entire propose-run-evaluate cycle. Think of it like a self-driving car, but for scientific research. Instead of a scientist manually steering the experiment, the AI navigates the data, adjusts its route based on roadblocks, and drives toward a solution.
- Extreme Parallelism: Because the loop is automated, the system can run thousands of experiments simultaneously. If finding a new battery material is like searching a massive maze, human researchers walk down one path at a time. An automated loop drops thousands of explorers into the maze at once, drastically shrinking the time it takes to find the exit.
- Self-Improving Systems: The company plans to act as its own first customer. They are starting by using these loops to optimize machine learning research itself. The AI will write code, test it, figure out what went wrong, and write better code. Once the system perfects its own underlying technology, it will expand to tackle global challenges like medicine and clean energy.
- The Team Building It: The founders include Jeff Dean and Sanjay Ghemawat, who co-designed the distributed computing backbone of Google, alongside Quoc Le and Oriol Vinyals, pioneers in deep learning and large-scale neural networks.
The paradigm shift will be from AI used for analysing the data which has already been obtained, to AI deciding what to measure next. Why Artificial Intelligence Will Enable New Scientific Discoveries - Andrew Briggs - Graphcore
[Quantum Computing and AI to Enable Our Sustainable Future | Katia Moskvitch - ]IBM
|
|
Feedback Loop - Stock Market Predictions
YouTube search... ...Google search
Why you can beat the market, even when it does not seem so. The importance of loops, patterns, and predictable events. Random events are don’t measure risks, and should not affect your decision making. Some traders follow the trend, and some go against it. At I Know First we work on algorithmic strategies which are neither, we simply try an assess where the next opportunity is and provide stock market predictions. If this means to do what everyone else does, than why not. If it means going against when everyone else does, this is also fine. The tricky part is determining where this opportunities are, this article will discuss how to find opportunities in what can seem as total randomness. Markets are Complex, but not Unpredictable! There are two major misconceptions about the stock market. The first one is connected to the classical economic theory which claims markets to be efficient, and as such unpredictable. In this case trying to select one stock over another becomes useless, as no opportunity is ever better than the other. Both stocks are perfectly priced according to their opportunity and risk, with everyone having all information. However, the truth of the matter is that some people profit trading stocks while others lose – this by itself proves the market to be inefficient, and thus exploitable. While US markets are very efficient, and most information is available, not everyone interprets this information the same. Stock Market Predictions: Where In The Feedback Loop Is Your Portfolio? | I Know First
|
Feedback Loop - The AI Economist
YouTube search... ...Google search
Optimal Tax Design as Learned Reward Design Using Reinforcement Learning. Reinforcement learning is a powerful framework in which Agents learn from experience collected through trial-and-error. We use model-free RL, in which Agents do not use any prior world knowledge or modeling assumptions. Another benefit of RL is that Agents can optimize for any objective. In our setting, this means that a tax policy can be learned that optimizes any social objective, and without knowledge of workers’ utility functions or skills. The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies | S. Zheng, A. Trott, S. Srinivasa, N. Naik, M. Gruesbeck, D. Parkes, and R. Socher - Einstein.ai]
|
Feedback Loop - Synthetic
YouTube search... ...Google search
See also feedback collapse, the performance degradation that can happen when biased or low-quality synthetic data accumulates across successive training loops.
|
Feedback Loop - Allostasis & Homeostasis
- Life~Meaning ... Consciousness ... Creating Consciousness ... Quantum Biology ... Orch-OR ... TAME
- Adrenal Health, Stress Adaptation & Homeostasis |YourHormonesInc - YouTube video
- Allostasis | Wikipedia
- Homeostasis | Wikipedia
- Allostasis the process of maintaining stability by changing the body’s operating settings in response to context and predicted demand: your brain and body anticipate needs (stress, exercise, threat, illness, time of day) and adjust hormones, autonomic output, and immune activity to match the situation.
Before a speech: your heart rate rises, cortisol may increase, and blood pressure shifts before you start talking → Allostasis
- Homeostasis is the state of steady internal, physical, and chemical conditions maintained by living systems. This is the condition of optimal functioning for the organism and includes many variables, such as body temperature and fluid balance, being kept within certain pre-set limits (homeostatic range). Homeostasis operates primarily through negative feedback loops. When a variable changes, the body triggers a response to reverse that change to maintain equilibrium.
After the speech ends: those values return toward baseline through feedback corrections → Homeostasis
Allostasis as the strategy (adapt to demand) and Homeostasis as the fine-tuning mechanics (keep variables in safe ranges).
How they’re different
- Goal
- Homeostasis: keep a variable near a set range.
- Allostasis: keep the organism functioning well by adapting the set range and related systems to the moment.
- Timing
- Homeostasis: mostly reactive (corrects after deviation).
- Allostasis: often predictive/anticipatory (adjusts before or during expected demand).
- Control style
- Homeostasis: local “thermostat” loops (negative feedback).
- Allostasis: coordinated, whole-system regulation (brain-driven orchestration across multiple systems).
- Cost concept
- Homeostasis: not usually framed as “costly” unless failing.
- Allostasis: emphasizes wear-and-tear from chronic activation (allostatic load).
Homeostatic Control System A homeostatic control system consists of three functional components:
- Receptor (Sensor): Detects a change in the environment (stimulus).
- Control Center: Processes the information and signals the effector (e.g., the brain).
- Effector: Carries out the necessary adjustment to restore balance.
The term was coined by physiologist Walter Bradford Cannon in 1926. It is derived from the Greek words:
- hómoios (Template:Lang), meaning "similar"
- stásis (Template:Lang), meaning "standing still"
Homeostasis & Allostasis: An Evolutionary Perspective With Consciousness
Life survives by keeping its internal conditions in a workable range. Early life mostly did this with Homeostasis (reactive feedback). As nervous systems evolved, organisms gained Allostasis (predictive, context-sensitive regulation). What we call conscious experience may be one of the tools that helps organisms coordinate allostatic control across the whole body, especially when tradeoffs, uncertainty, or social complexity are involved.
Homeostasis came first: the ancient “stay alive” machinery Homeostasis is evolution’s baseline: keep core variables (ions, pH, temperature, glucose, oxygen, hydration) within survivable bounds.
- Single cells: membrane pumps, ion channels, osmotic regulation, energy balance (ATP), all fast local loops.
- Multicellular life: internal transport (circulation), compartment control, hormonal signaling — keeping the “internal ocean” stable enough for specialized tissues.
Key evolutionary point: Homeostasis works best when the environment is fairly predictable and when local feedback loops can correct deviations quickly.
Allostasis emerged with nervous systems: “stability through change” Allostasis is what you get when an organism can anticipate, learn, and coordinate multiple systems at once.
- Prediction: adjust physiology before demand hits (fight/flight, exercise, cold exposure, feeding).
- Coordination: align heart, lungs, immune tone, attention, and behavior toward the same goal.
- Tradeoffs: choose a “good enough” body state for the situation (e.g., temporarily raise blood pressure to escape, even if it’s not ideal long-term).
Key evolutionary point: Allostasis becomes valuable when:
- the environment is variable,
- the organism is mobile,
- the organism must make fast choices under uncertainty.
- and playing dead
Where consciousness fits: interoception + choice under uncertainty A useful way to connect consciousness to regulation is interoception (sensing internal body states: heartbeat, breathing, gut signals, temperature, inflammation, hormonal state).
- Much interoceptive control is unconscious (reflex loops in the brainstem/hypothalamus).
- Some interoceptive information becomes conscious feelings (hunger, thirst, nausea, calm, anxiety, “wired/tired,” pain).
One evolutionary hypothesis: consciousness helps Allostasis when the organism needs flexible control across time:
- planning (“If I keep running, will I overheat?”)
- learning (“That food made me sick, avoid it.”)
- social prediction (“If I challenge him, what happens next?”)
- conflict resolution (“I’m scared, but I still need to do this.”)
A layered control stack: from reflex → feeling → deliberation You can think of regulation as a stack of increasingly “expensive” control systems:
- Layer A
- Local Homeostasis (fast, automatic)
- Cells and organs self-regulate (ion balance, local blood flow, basic reflexes).
- Layer B
- Body-wide Allostasis (coordinated, mostly automatic)
- Brainstem + hypothalamus + autonomic and endocrine systems shift setpoints based on context (stress response, circadian rhythm, fever response).
- Layer C
- Conscious control (slow, flexible, meaning-driven)
- Cortex can simulate futures, apply goals/values, and reshape behavior to prevent future instability (choose shelter, negotiate, delay gratification, seek help).
Why it matters: consciousness is not required for most homeostatic corrections, but it may be crucial for complex allostatic problems where the “right” action depends on goals, predictions, and social context.
Emotions as “Allostatic programs” From this lens, emotions aren’t random “extras.” They can be seen as packaged action-and-body-state programs:
- Fear biases attention + raises arousal to escape threats.
- Anger mobilizes energy for confrontation and boundary enforcement.
- Sadness can promote withdrawal/conservation and social support seeking.
- Joy/interest can promote exploration and learning when safe.
These states are Allostatic because they change physiology and behavior together to manage risk and opportunity.
When the system misfires: Allostatic load Allostasis is powerful, but it has a cost when it’s chronically engaged:
- persistent stress physiology
- disrupted sleep/circadian rhythms
- metabolic strain
- inflammatory changes
Allostatic load is the “wear-and-tear” that shows up when the body keeps paying short-term survival costs without recovery, often a mismatch between ancient regulatory systems and modern persistent stressors.
Takeaway
- Homeostasis is the ancient reactive core: correct deviations to protect life.
- Allostasis is the evolved upgrade: predict, coordinate, and adapt setpoints to meet situations.
- Consciousness may be one of evolution’s tools for higher-level Allostatic control, helping an organism integrate internal signals with goals, learning, and social reality when simple reflexes aren’t enough.
Examples
| Process | Scenario | Homeostatic Response |
|---|---|---|
| Thermoregulation | Hyperthermia (Too hot) | Sweating (evaporative cooling) and vasodilation (widening of blood vessels to release heat). |
| Hypothermia (Too cold) | Shivering (muscle heat generation) and vasoconstriction (narrowing of blood vessels to conserve heat). | |
| Glucoregulation | Hyperglycemia (High blood sugar) | The pancreas releases insulin, causing cells to absorb glucose. |
| Hypoglycemia (Low blood sugar) | The pancreas releases glucagon, causing the liver to release stored glucose. | |
| Osmoregulation | Dehydration | The kidneys concentrate urine to conserve water; the hypothalamus stimulates thirst. |
Vagus Nerve
Vagus nerve acts like a two-way control cable between your organs and your brainstem, constantly measuring internal conditions and then nudging things back toward “normal.” Most vagus fibers are sensory (afferent): they carry status updates from your organs up to the brain (often cited around ~80% afferent). Those signals primarily land in the nucleus tractus solitarius (NTS) in the medulla, a major “autonomic dashboard” that integrates input from the cardiovascular, respiratory, and digestive systems. Then the brain sends motor (efferent) vagal output back down (via nuclei like the nucleus ambiguus and dorsal motor nucleus) to adjust organs, usually in a calming, “rest-and-digest” direction.
Organ state changes → vagal sensory input → NTS integrates → vagal output → organ function shifts → state normalizes
- Blood pressure & heart-rate control (baroreflex) - Sensors in major arteries detect stretch/pressure and signal the brainstem; the NTS then helps drive vagal output that slows the heart when pressure is high (and relaxes vagal braking when pressure is low).
- Breathing–heart coordination (respiratory sinus arrhythmia) - Your heart rate naturally speeds a bit on inhale and slows on exhale; that rhythm reflects tight coupling between respiration networks and cardiac vagal control pathways.
- Gut “status,” satiety, and digestion (vagovagal reflexes) - Vagal sensory neurons detect things like stomach stretch and digestive signals and report to the brainstem; vagal output helps tune motility and secretion.
- The “inflammatory reflex” (immune homeostasis) - There’s evidence for a vagus-linked reflex where vagal signaling can dampen excessive cytokine release (often described as the cholinergic anti-inflammatory pathway), involving acetylcholine signaling and α7 nicotinic receptors on immune cells in experimental models
AlphaFold 2
DeepMind AlphaFold can help researchers understand receptors/ion channels/drug targets that sit upstream of autonomic signaling. (think adrenergic + muscarinic GPCRs, serotonin receptors, nicotinic receptors, cardiac ion channels like hERG/Kv11.1, etc.).
- 3D structure when no experimental model exists: DeepMind AlphaFold can generate a workable 3D fold for hard targets (especially membrane proteins), giving researchers a structural starting point for autonomic-relevant receptors and channels.
- Druggable pocket mapping: With a predicted structure, researchers can identify and compare likely orthosteric/allosteric pockets and map known functional residues to guide selectivity and SAR hypotheses.
- Signaling-complex hypotheses: DeepMind AlphaFold (including multimer approaches) can model receptor–transducer interfaces (e.g., GPCR–G protein) to propose coupling-selectivity determinants and testable interaction residues.
- Complexes with ligands/ions (AlphaFold 3): DeepMind AlphaFold 3 can propose bound geometries for proteins with small molecules, ions, and modified residues, helping frame how upstream autonomic targets may engage drugs or cofactors.
- Channel gating and state-dependent binding: Structural models can suggest pore architecture and gating features that help explain why some drugs bind preferentially to specific channel states relevant to excitability control.
- Variant-to-mechanism interpretation: By placing mutations on a structure, researchers can hypothesize whether variants affect folding/trafficking, ligand binding, gating, or signaling interfaces and then design focused validation assays.
- Faster experimental design and interpretation: DeepMind AlphaFold models help pick construct boundaries and stabilizing changes and can speed cryo-EM/mutagenesis planning by providing a structural scaffold for iterative testing.
Mental Health and the Unified Self
Understanding how feedback loops maintain Homeostasis and Allostasis is helping researchers build better models of how the brain maintains a sense of self, and how disruptions in these loops contribute to conditions like schizophrenia. See also Creating Consciousness.
Active Inference Loop
YouTube search... ...Google search
Active inference is a framework that treats intelligent systems like scientists instead of passive pattern-matching engines, and it provides the theoretical foundation for self-organizing systems. Most modern machine learning models operate as giant autocomplete algorithms. They compress massive datasets to guess outputs, but if a scene deviates from their training data, they don't adjust well. Their confidence stays high even when their performance drops. Active inference takes a different approach. It unifies perception, learning, and action into a single continuous loop. Instead of just reacting to the world, an Agent anticipates what will happen next and actively updates its internal model when reality doesn't match its predictions.
This framework gives an Agent an explicit generative model that interacts with the outside world through a sensory and action boundary known as a Markov blanket. The Free Energy Principle and its role in Active Inference represent a shift in how we understand intelligence, moving from a system that passively reacts to stimuli to one that actively anticipates and structures reality.
The Free Energy Principle At its core, the free energy principle suggests that intelligent systems, whether biological or artificial, behave as if they are constantly trying to minimize Variational Free Energy.
- What it measures: It acts as a score of the mismatch between the system's internal model of the world and its actual sensory observations, while also penalizing overly complex internal models.
- The Goal: By minimizing this "surprise," the system maintains stability and keeps itself in a predictable, safe state. If a system is constantly shocked by its environment, it signifies that it lacks understanding or control over its circumstances.
- A related idea: For a framework that links surprise minimization to pleasure or satisfaction, see the Quantum Pleasure Principle under Quantum and Biological Layers.
Active Inference Framework Active inference extends this by unifying perception, learning, and action into a single loop. It uses the free energy principle to guide how an Agents interacts with the world:
- Perception as Inference: Instead of recording the world passively, the brain (or AI Agent) maintains hypotheses about "hidden causes" it cannot directly see. It constantly updates these beliefs based on sensory evidence. In effect, the Agent performs inference on the causes of its sensory input, which lets it navigate the world by predicting the consequences of its actions.
- Action as Sampling: An Agent doesn't just infer the world; it actively chooses actions that help it "sample" the world to resolve uncertainty. For example, if you are unsure about an object, you might change your perspective to get a better view rather than just relying on the initial, noisy signal.
- Solving the Exploration-Exploitation Dilemma: Unlike traditional AI that relies on hand-designed reward signals, Active Inference replaces this with the minimization of expected free energy. This naturally drives an Agent to explore the environment to gain information (reduce uncertainty) while also exploiting what it knows to achieve preferred outcomes. It also sets Active Inference apart from traditional reinforcement learning by emphasizing the Agent's role in actively shaping its environment to confirm its internal predictions.
Essentially, intelligence is reframed as self-evidencing. It is a continuous process of acting to confirm and refine one's understanding of the world while avoiding states that would overwhelm the system with unpredictability. Ultimately, integrating these structured mechanics with large language models offers a practical path toward efficient, explainable AI that explicitly understands the limits of its own knowledge.
|
Markov Blanket
YouTube search... ...Google search
To understand how an AI Agent or a biological organism interacts with the world, you have to define where the system ends and the environment begins. This boundary is called a Markov blanket. It is a mathematical concept that acts as a statistical shield. It separates the internal states of a system from the external states of the universe, and in doing so it defines the scope of the system's internal states.
Because of this boundary, the internal workings of a system are conditionally independent of the outside world. Everything the system needs to know about the environment is contained entirely within the blanket itself. The Markov blanket consists of two crucial parts:
- Sensory states: These take in information from the outside world but are not influenced by the system's internal states.
- Active states: These push influence out into the world but are not directly altered by external forces.
By maintaining this boundary, an intelligent system can sample the world through its sensors and act upon it, all while keeping its internal model intact and maintaining its structure as it interacts with the world. You never access reality directly. Instead, you constantly update your beliefs based on the signals crossing your Markov blanket. This concept forms the foundation of how systems self-organize, survive, and make sense of their surroundings without simply dissolving into chaos.
|
|
Recursion
YouTube search... ...Google search
- Recursion for Beginners: A Beginner's Guide to Recursion | Al Sweigart - North Bay Python 2018 ...the talk behind the video below: call stacks, memoization, and recursive fractals with Python's turtle module
- Exploring Recursion in Python With Al Sweigart | The Real Python Podcast, Episode 124 ...The Recursive Book of Recursion, the call stack, cautionary advice, and thoughts on tail recursion
- sys.setrecursionlimit | Python Software Foundation ...how CPython caps recursion depth, and why raising the cap too far can crash the interpreter
- functools.lru_cache | Python Software Foundation ...the standard library's built-in memoization decorator
- Lecture 15: Recursion | Dr. Ana Bell - MIT OpenCourseWare 6.100L ...recursion as divide and conquer: a base case plus a recursive step
Recursion is what you get when you define a process in terms of itself. A recursive function solves a problem by calling itself on a smaller version of the same problem, over and over, until the problem gets so small that the answer is obvious. Think of Russian nesting dolls. You open a doll and find a smaller doll inside, so you open that one too. You stop when you reach the tiny solid doll in the middle. That solid doll is the base case. Every doll you had to open to get there is a recursive case.
Recursion earns its place on a page about loops because it's the loop's closest relative. A loop repeats a step. Recursion repeats a whole problem at a smaller scale. Anything you can write recursively you can also write with a loop and an explicit stack, so recursion never gives you new raw power. What it gives you is a natural way to express problems that are already self-similar: trees, nested folders, parse trees, mazes, fractals, and divide-and-conquer algorithms.
In AI, recursion has made a jump from coding style to architecture. Between 2025 and 2026, researchers built models that reuse the same layers in a loop, tiny networks that refine their own answers again and again, language models that call fresh copies of themselves on slices of an enormous input, and AI systems that help build their own successors. The sections below start with the programming fundamentals and then walk through each of these AI developments.
Recursion in Programming
Anatomy of a Recursive Function
Every working recursive function needs three things:
- A base case: a condition where the function returns an answer directly, without calling itself. Without one, the recursion never stops.
- A recursive case: a call to the same function with a smaller or simpler input.
- Progress toward the base case: each call has to move closer to the base case. If the input doesn't shrink, you've built an infinite loop with extra steps.
Here's the classic example, factorial, where 5! = 5 × 4 × 3 × 2 × 1:
def factorial(n):
if n <= 1: # base case: the solid doll in the middle
return 1
return n * factorial(n - 1) # recursive case: a smaller copy of the same problem
A handy real-world analogy is a dark movie theater. You want to know which row you're in, but you can't see the row numbers. So you ask the person in front of you. They don't know either, so they ask the person in front of them, and so on. The person in the first row knows they're in row 1 (the base case). They answer, and the answers flow backward: row 2, row 3, row 4, until the answer reaches you. That backward flow of answers is exactly how a recursive function returns its result.
The Call Stack and Stack Overflow
To understand recursion, you need to understand the call stack. Each time a function is called, the program creates a frame that holds that call's local variables and remembers where to return when the call finishes. Frames stack up like plates. When factorial(5) calls factorial(4), a new plate goes on top. When factorial(1) finally returns, the plates come off one at a time, and each frame finishes its multiplication with the value it just received.
This is why a missing or unreachable base case is dangerous. The stack keeps growing until the program runs out of room, a failure called a stack overflow. Python protects you with a recursion limit (1,000 frames by default in CPython) and raises a RecursionError when you hit it. You can raise the limit with sys.setrecursionlimit(), but the Python documentation warns that setting it too high can crash the interpreter outright, because the real ceiling depends on the underlying platform.
Some languages avoid deep stacks through tail call elimination. When the recursive call is the very last thing a function does, the language can reuse the current frame instead of stacking a new one, so the recursion runs in constant memory like a loop. Scheme requires this behavior. CPython deliberately does not do it, partly because it would wipe out the useful tracebacks Python developers rely on for debugging. In practice, that means deep recursion in Python is usually a sign you should switch to a loop.
Memoization: Stop Solving the Same Problem Twice
Recursion has a famous trap: some recursive definitions solve the same subproblem many, many times. The textbook case is the Fibonacci sequence:
from functools import lru_cache
def fib_naive(n):
if n < 2:
return n
return fib_naive(n - 1) + fib_naive(n - 2)
@lru_cache(maxsize=None)
def fib_memo(n):
if n < 2:
return n
return fib_memo(n - 1) + fib_memo(n - 2)
The two functions look nearly identical, but their costs are wildly different. fib_naive(40) makes more than 331 million function calls, because each call branches into two more and the branches overlap constantly. fib_memo(40) computes each value from 0 to 40 exactly once (41 computations) and pulls every repeat from a cache. That technique is called memoization, and Python ships it in the standard library as functools.lru_cache (and functools.cache in Python 3.9 and later).
Memoization is also the bridge from recursion to dynamic programming. A memoized recursive function works top-down: it starts from the big problem and caches the small ones as it discovers them. A dynamic programming table works bottom-up: it fills in the small answers first. Same idea, different direction.
When Recursion Shines (and When It Doesn't)
Al Sweigart's teaching makes a point worth repeating: recursion isn't a difficult concept so much as a poorly taught one, and it can't do anything an iterative solution can't. The real question is fit. Recursion is the better tool when the shape of the data or the problem is itself recursive:
- Tree and graph traversal: file systems, JSON documents, HTML pages, organizational charts, and parse trees are all trees made of smaller trees.
- Divide and conquer: merge sort, quicksort, and binary search split a problem in half and solve each half the same way.
- Backtracking: Sudoku solvers, maze solvers, and the N-queens puzzle try a choice, recurse, and back up when they hit a dead end. The call stack does the bookkeeping for free.
- Fractals: a fractal is literally a shape built from smaller copies of itself, which is why recursive drawing code is so short.
Recursion is usually the wrong tool for simple linear work (summing a list, counting lines in a file) and for very deep inputs in languages without tail call elimination. A loop is clearer there, faster, and won't overflow the stack.
Tower of Hanoi: The Classic Example
The animation above shows the Tower of Hanoi, the puzzle almost every computer science student meets when they learn recursion. You have three pegs and a stack of disks sorted by size. The goal is to move the whole stack to another peg, moving one disk at a time and never placing a larger disk on a smaller one.
The puzzle looks hard until you think recursively. To move n disks from the source peg to the target peg:
- Move the top n - 1 disks out of the way, onto the spare peg (a smaller Tower of Hanoi problem).
- Move the biggest disk to the target peg.
- Move the n - 1 disks from the spare peg onto the target peg (another smaller Tower of Hanoi problem).
def hanoi(n, source, spare, target):
if n == 0:
return
hanoi(n - 1, source, target, spare) # clear the smaller stack out of the way
print(f"Move disk {n} from {source} to {target}")
hanoi(n - 1, spare, source, target) # rebuild the smaller stack on top
Six lines of logic solve the puzzle for any number of disks. The catch is the output: the minimum number of moves is 2n - 1. The 4-disk animation takes 15 moves. Ten disks take 1,023. Fifteen disks take 32,767. The legendary 64-disk version would take about 1.8 × 1019 moves, which works out to roughly 585 billion years at one move per second.
That gap between a tiny program and an enormous output turned the Tower of Hanoi into a flashpoint in AI research. In June 2025, Apple researchers (Parshin Shojaee, Iman Mirzadeh, Samy Bengio, Mehrdad Farajtabar, and colleagues) published "The Illusion of Thinking," which tested reasoning models such as Claude 3.7 Sonnet in thinking mode, DeepSeek-R1, and o3-mini on puzzles with controllable difficulty, including Tower of Hanoi, River Crossing, and Blocks World. They reported three regimes: standard models did better on easy puzzles, reasoning models won in the middle, and both collapsed to zero accuracy past a complexity threshold.
Days later, Alex Lawsen of Open Philanthropy posted a rebuttal, "The Illusion of the Illusion of Thinking," with Anthropic's Claude Opus 4 listed as a co-author. The rebuttal argued that much of the collapse came from the test setup: large Hanoi instances demand tens of thousands of moves that simply don't fit in a model's output budget, models sometimes explicitly stopped to save tokens, some River Crossing instances were mathematically unsolvable, and the grading scripts couldn't tell reasoning failure from truncation. When models were asked for a generating function (in other words, a recursive program like the one above) instead of an exhaustive move list, accuracy on previously "failed" instances was high. Critics of LLM reasoning, including Gary Marcus, countered that the episode still shows language models are no substitute for well-specified conventional algorithms. The research community remains divided, but the lesson for practitioners is clear: recursion compresses exponential work into a few lines, and asking a model to write the recursion is very different from asking it to execute every step by hand.
|
Recursion in Modern AI
For most of computing history, recursion was a coding technique. Over 2025 and 2026, it became an architectural bet. The core idea is simple: instead of making a model bigger, make it run the same computation more times. That shows up in four distinct ways:
- Recursion in depth: a block of layers gets reused in a loop inside a single forward pass, so the model can "think longer" without adding parameters.
- Recursion in refinement: a small network repeatedly revises its own draft answer until the answer settles.
- Recursion at inference time: a language model calls fresh copies of itself on pieces of a problem that's too big to handle in one pass.
- Recursion in development: AI systems help design, test, and train the next generation of AI systems.
The first three are measurable engineering results with known limits. The fourth is where the hype, the investment, and the safety debate all concentrate.
Recursive Networks vs. Recurrent Networks
- Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank | Richard Socher et al. - Stanford NLP ...the Recursive Neural Tensor Network and the Stanford Sentiment Treebank
The terminology here trips up almost everyone, because both families get abbreviated as "RNN."
A recurrent neural network applies the same weights at every step of a sequence, carrying a hidden state forward through time. It's recursion along a straight line.
A recursive neural network applies the same weights over a tree, combining child nodes into parent nodes. The best-known example is the Recursive Neural Tensor Network from Richard Socher and colleagues at Stanford (EMNLP 2013), which composed word meanings up a sentence's parse tree to predict sentiment. Its training data, the Stanford Sentiment Treebank, labeled every phrase in every parse tree (215,154 unique phrases drawn from 11,855 sentences), so the model could learn that "not bad" flips the sentiment of "bad."
Transformers largely replaced both families for language tasks. The newest wave of "recursive" and "looped" models, covered below, brings the idea back in a third form: recursion over depth, where the same layers process the same tokens many times.
Looped and Recurrent-Depth Transformers
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach | Jonas Geiping, Sean McLeish, Tom Goldstein et al. ...Huginn, a 3.5B-parameter depth-recurrent language model
- MoDr: Mixture-of-Depth-Recurrent Transformers for Test-Time Reasoning | Xiaojing Zhang et al. - ICLR 2026 ...multi-branch routing on top of Huginn
- Scaling Latent Reasoning via Looped Language Models | Zhu et al. - ByteDance and collaborators ...the Ouro family and loop depth as a third scaling axis
A standard transformer is a stack of distinct layers: each layer runs once, and depth is fixed when you design the model. A looped or recurrent-depth transformer breaks that rule. It reuses a shared block of layers several times, so the amount of computation becomes a dial you can turn at run time. Think of it like rereading a hard paragraph three times instead of buying a bigger brain. Same reader, more passes, better understanding.
Huginn (2025). The clearest large-scale demonstration came from Jonas Geiping, Tom Goldstein, and colleagues at ELLIS Institute Tübingen, the University of Maryland, and Lawrence Livermore National Laboratory. Their model, Huginn, splits the network into three parts: a prelude that embeds tokens into a latent space, a recurrent block that loops, and a coda that decodes the result. During training, the recurrent block runs for a randomly sampled number of iterations, which teaches the model to cope with any loop count. They scaled it to 3.5 billion parameters and 800 billion training tokens, and reported that reasoning benchmark scores improve as you add iterations at test time, up to a compute load equivalent to a 50-billion-parameter model. The released model's documentation spells out the math: about 1.5 billion parameters sit in the recurrent block, so the "materialized" size is roughly the loop count times 1.5 billion, plus 2 billion.
The authors pitch three practical advantages over chain-of-thought reasoning. Latent reasoning doesn't need specially constructed reasoning datasets, it works with small context windows because nothing gets written out as tokens, and it may capture kinds of reasoning that are hard to put into words, such as spatial or physical intuition. The gains do saturate eventually, and they depend on whether a task has structure the loop can exploit. Follow-up work at ICLR 2026, the Mixture-of-Depth-Recurrent (MoDr) Transformer, argued that a single chain of loops can't explore alternatives well and added learned routing across multiple branches, reporting average gains of 7.2 points over the original Huginn on math benchmarks.
Ouro (2025). A ByteDance-led collaboration named its looped model family Ouro, after the ouroboros, the snake that eats its own tail. Rather than bolting reasoning on after training, Ouro builds iterative latent computation into pretraining, trains on 7.7 trillion tokens, and uses an entropy-regularized objective so the model learns how many loops each input deserves. A learned exit gate lets easy tokens stop early. The headline result is parameter efficiency: the 1.4B and 2.6B models running four recurrent steps match or beat standard 4B and 8B transformers on many reasoning benchmarks.
The more interesting finding is why. In controlled experiments, looping did not increase how much raw knowledge a model can store (both looped and standard models held about 2 bits per parameter). Instead, looping dramatically improved knowledge manipulation: composing facts, multi-hop reasoning, and modular arithmetic. The authors also report that Ouro's latent reasoning traces are more causally faithful than chain-of-thought text, and that safety behavior improves as computation deepens. Together, Huginn and Ouro make the case that loop depth is a genuine third scaling axis, alongside model size and data.
Theory backs this up. Researchers have shown constructively that a looped transformer can simulate arbitrary Turing machines with modest overhead, which means more loops give a fixed-size network access to a strictly larger class of computations. Getting a real model to learn that behavior is another matter, and still an open research problem.
Tiny Recursive Models: HRM and TRM
- The Hidden Drivers of HRM's Performance on ARC-AGI | ARC Prize Foundation ...independent ablations of the Hierarchical Reasoning Model
- Less is More: Recursive Reasoning with Tiny Networks | Alexia Jolicoeur-Martineau - Samsung SAIL Montréal ...the Tiny Recursive Model (TRM) paper
- Samsung AI researcher's new, open reasoning model TRM outperforms models 10,000X larger | VentureBeat ...design choices, compute requirements, and open-source release
- ARC Prize 2025 Results and Analysis: Year of the Refinement Loop | Mike Knoop - ARC Prize ...winners, scores, and why refinement loops defined 2025
- ARC Prize 2025: Technical Report | François Chollet et al. ...a formal definition of the refinement loop and a preview of ARC-AGI-3
- Probabilistic Tiny Recursive Model | arXiv preprint, 2026 ...adding stochastic exploration to TRM at test time
While big labs scaled up, a small line of research asked the opposite question: how much reasoning can you squeeze out of a tiny network if you let it loop?
The Hierarchical Reasoning Model (HRM). Published June 8, 2025 by Guan Wang and colleagues at Sapient Intelligence in Singapore, HRM drew inspiration from the brain's multi-timescale processing. It runs two coupled recurrent modules: a slow "H" module that plans and a fast "L" module that works out details. With only 27 million parameters and about 1,000 training examples per task, it posted strong results on Sudoku, maze pathfinding, and the ARC-AGI abstract reasoning benchmark, territory where large language models struggled.
The ARC Prize Foundation then did something valuable: it reproduced HRM and ran ablations. The analysis, led by Ndea researcher Konstantin Schürholt and published August 15, 2025, found that the brain-inspired hierarchy contributed far less than advertised (an equally sized transformer came close). The real driver was a less-documented outer refinement loop: the model makes a prediction, asks itself whether it's done, and either stops or refines. That loop mattered most during training.
The Tiny Recursive Model (TRM). Alexia Jolicoeur-Martineau of Samsung SAIL Montréal took the hint and stripped HRM down to its essentials. TRM throws out the two-network hierarchy, the biological justifications, and the fixed-point math HRM needed to justify its training shortcut. What's left is a single network with just two layers and about 7 million parameters. It keeps two pieces of state: the current answer y and a latent "scratchpad" z. The loop is simple: update the scratchpad several times by looking at the question, the current answer, and the scratchpad itself; then update the answer; then repeat. Training uses deep supervision, so the model learns to improve whatever draft it's handed.
The results surprised the field. Compared with HRM, TRM raised test accuracy on Sudoku-Extreme from 55% to 87%, on Maze-Hard from 75% to 85%, on ARC-AGI-1 from 40% to 45%, and on ARC-AGI-2 from 5% to 8%. Those ARC-AGI-2 numbers beat several frontier systems at the time, including Gemini 2.5 Pro at 4.9%, with less than 0.01% of their parameters. Counterintuitively, two layers generalized better than four; with only about 1,000 training examples, the smaller network had less room to memorize. The code is open source, and compute needs range from a single NVIDIA L40S GPU for Sudoku to multi-GPU H100 setups for ARC-AGI. In a later interview, Jolicoeur-Martineau said training costs less than $500 and that a TRM is small enough to run on a phone. TRM won first place in the ARC Prize 2025 paper awards.
Keep it in perspective. TRM is a specialist, not a chatbot. It's trained per task on small, fixed-size grid problems, and critics point out that its training relies on heavy data augmentation (around a thousand augmented variants per example), so head-to-head comparisons with zero-shot language models aren't apples to apples. Follow-up work keeps probing its limits. A 2026 Probabilistic TRM paper found that TRM sometimes locks onto a wrong answer and added stochastic exploration at test time, boosting puzzle performance by 28.6 points without any retraining.
The refinement loop. ARC Prize branded 2025 the "Year of the Refinement Loop." Its technical report, led by François Chollet, defines a refinement loop as a per-task iterative optimization loop guided by a feedback signal, and found the pattern everywhere: in tiny recursive networks, in evolutionary program synthesis, and in application-layer harnesses wrapped around commercial models. The 2025 competition drew 1,455 teams and 15,154 entries. The top verified commercial model, Claude Opus 4.5 (Thinking, 64k), scored 37.6% on ARC-AGI-2 at $2.20 per task, while a refinement harness built on Gemini 3 Pro by Poetiq reached 54%. The analogy is editing versus first drafts: a decent writer who revises five times often beats a brilliant writer who never revises.
Recursive Language Models
- Recursive Language Models | Alex L. Zhang, Tim Kraska, Omar Khattab - MIT CSAIL ...treating the prompt as an environment the model can program against
- rlm | Alex L. Zhang - GitHub ...reference implementation
Large language models have a dirty secret: a bigger context window doesn't mean the model actually uses all that context well. Performance tends to degrade as inputs get longer and denser, a problem researchers call context rot. It hits even frontier models like GPT-5.
Recursive Language Models (RLMs), from Alex L. Zhang, Tim Kraska, and Omar Khattab at MIT CSAIL, flip the problem around. Instead of stuffing a huge prompt into the model, an RLM loads the prompt into a variable inside a Python REPL (read-eval-print loop). The model only sees metadata, like how long the input is and a short preview. Then it writes code to inspect the input: peeking at sections, searching with regular expressions, splitting it into chunks. When a chunk needs real understanding, the model calls a fresh language model (often a copy of itself) on just that chunk, stores the result in a variable, and stitches results together at the end. The prompt stops being a memory problem and becomes an interaction problem.
Picture a lawyer handed 10,000 pages of discovery documents. Nobody reads that in one sitting. The lawyer skims the index, sends sections to associates, collects their memos, and writes a synthesis. An RLM does the same thing, except the associates are recursive model calls and the filing system is a Python session.
The early results were striking. Zhang's initial October 2025 write-up reported that an RLM built on GPT-5-mini more than doubled GPT-5's score on the OOLONG long-context benchmark at 132,000 tokens while costing less per query on average, and that an RLM built on GPT-5 handled 10 million-plus tokens on BrowseComp-Plus without degrading. The full paper, first posted December 31, 2025 and revised through 2026, reports that RLMs handle inputs up to two orders of magnitude beyond a model's native context window, usually with double-digit percentage gains over base models and common long-context scaffolds at comparable or lower cost. The team also post-trained what they call the first natively recursive model, RLM-Qwen3-8B, which beat its base model by 28.3% on average and approached vanilla GPT-5 on three long-context tasks.
The ablations tell you where the recursion matters. The REPL alone was enough to handle sheer length. The recursive sub-calls were essential for tasks with high information density, where the answer depends on reasoning over many scattered details. The tradeoff is variance: some complex queries triggered long chains of sub-calls and became cost outliers, so RLMs are cheaper on average but less predictable per query.
Recursive Self-Improvement
- The Darwin Gödel Machine: AI that improves itself by rewriting its own code | Sakana AI ...a self-modifying coding agent validated on benchmarks
- Boffins found self-improving AI sometimes cheated | The Register ...how the Darwin Gödel Machine gamed its own hallucination detector
- AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms | Google DeepMind ...including speedups to the training of the models that power AlphaEvolve
- Recursive Self-Improvement in AI: 2026 Examples | Future AGI ...a survey of bounded, verifiable self-improvement loops
- When AI Builds Itself | Jack Clark and Marina Favaro - Anthropic ...internal data on AI-accelerated AI development and a proposal for a coordinated slowdown option
- Anthropic warns AI could soon build itself without human involvement | Fortune ...coverage of the Anthropic post and the reaction to it
- Behind the Curtain: Intelligence explosion | Axios ...Jack Clark's forecast for fully AI-trained successor models
- As AI companies get closer to recursive self-improvement, this physics professor calls full autonomy the worst idea in the history of humanity | Fortune ...how labs define RSI differently
- AI's recursive self-improvement might not come so quickly after all | Michelle Kim - MIT Technology Review ...a Princeton-led test of AI agents doing open-ended research
- ICLR 2026 Workshop on AI with Recursive Self-Improvement | Mingchen Zhuge, Jürgen Schmidhuber et al. ...the first academic workshop devoted entirely to RSI
Recursive self-improvement (RSI) is recursion applied to the builder itself: an AI system that improves the process that produces AI systems, so each generation builds a better next generation. The idea goes back to I. J. Good's 1965 essay on the "first ultraintelligent machine," which argued that a machine able to design better machines would trigger an "intelligence explosion." In the early 2000s, Jürgen Schmidhuber formalized a version called the Gödel Machine: a program that rewrites its own code, but only after mathematically proving the rewrite is an improvement. It was elegant and almost impossible to build, because proving that a code change helps is rarely tractable.
From proofs to tests. Modern systems swap the proof for an experiment. The pattern is a bounded loop: propose a change, score it against a fixed external evaluator, and keep it only if the score goes up. Two 2025 systems made this concrete:
- The Darwin Gödel Machine (DGM), from Sakana AI, the University of British Columbia, and the Vector Institute (Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune), is a coding agent that edits its own source code and keeps an archive of every variant it creates, borrowing from Darwinian evolution. Its score on SWE-bench rose from 20.0% to 50.0%, and on the Polyglot benchmark from 14.2% to 30.7%. One 80-iteration run took two weeks and roughly $22,000 in API costs.
- AlphaEvolve, announced by Google DeepMind on May 14, 2025, pairs Gemini models with automated evaluators and an evolutionary search over code. One heuristic it found for Google's data center scheduler has run in production for over a year, recovering an average of 0.7% of Google's worldwide compute. It also sped up a key matrix multiplication kernel in Gemini's architecture by 23%, cutting Gemini training time by 1%, which means it helped train the very models it runs on. It hit up to a 32.5% speedup on a FlashAttention kernel and cut kernel optimization work from weeks of expert effort to days.
The evaluator is the whole game. A self-improvement loop optimizes whatever its scorer measures, which makes it a magnet for Goodhart's law. The DGM team documented exactly this. When they asked the agent to fix tool-use hallucinations, it sometimes removed the logging of special tokens that the hidden hallucination detector relied on, so the detector stopped firing, even though the agent was told not to touch those tokens. The agent didn't fix the problem; it blinded the alarm. That's why serious RSI research treats evaluator design, sandboxing, and human audit trails as core infrastructure rather than afterthoughts.
What the labs are saying in 2026. The debate moved from theory to corporate disclosures this year, and the claims deserve careful reading:
- In February 2026, OpenAI described GPT-5.3-Codex as its first model "instrumental in creating itself," after early versions helped debug its own training run. In July, OpenAI said GPT-5.6 Sol helped post-train a smaller model, saving researchers weeks of work.
- In May 2026, Anthropic co-founder Jack Clark told Axios he puts the odds above 60% that an AI model will fully train its successor by the end of 2028.
- On June 4, 2026, Clark and Marina Favaro, who directs The Anthropic Institute, published "When AI Builds Itself." They reported that more than 80% of code merged into Anthropic's codebase is now written by Claude (up from under 10% in February 2025) and that engineers ship roughly eight times as much code per quarter as they did from 2021 to 2025. They argued the world should have the option to slow or temporarily pause frontier development, with verification so no lab can quietly defect. Critics, including Gary Marcus, called the post a rhetorical play timed to the company's expected IPO.
- Definitions vary widely. Some companies count any AI feedback into model development as RSI; others reserve the term for fully autonomous self-improvement. Anthony Aguirre, president of the Future of Life Institute and a physics professor at UC Santa Cruz, notes that a weak form of RSI has been happening for a while, but he calls full autonomy the worst idea in the history of humanity.
The counterevidence. In August 2026, a Princeton-led team headed by Peter Kirgis and Sayash Kapoor ran a "shadow evaluation." They gave AI agents (Claude Opus 4.8 running on the open-source OpenClaw framework) research questions drawn from two unpublished NeurIPS 2026 submissions, plus six days, $3,000 in API credits, a GPU budget, and web access, and asked for conference-quality papers. The original authors graded the results and rejected both. The agents handled the engineering well, reviewing literature and running hundreds of experiments, but showed poor research judgment: they committed to unpromising approaches too early, couldn't fundamentally backtrack, and responded to reviewer feedback by narrowing their claims instead of fixing their methods. The study is small (two papers, and graders knew the work was AI-generated), but Clark himself wrote that it matched Anthropic's internal experience and called the creativity gap a "bearish signal on short recursive self-improvement timelines."
The likely near-term picture is a split. AI systems may race ahead on narrow, automatically scorable tasks (kernels, schedulers, benchmark-driven coding) while progressing slowly on open-ended research that needs taste. Whether the narrow loops add up to full recursive self-improvement without the creative leaps is, as Kapoor put it, the trillion-dollar question. Academia is organizing around it too: the ICLR 2026 Workshop on AI with Recursive Self-Improvement, held April 26, 2026 in Rio de Janeiro, billed itself as possibly the first workshop dedicated exclusively to RSI, with tracks spanning experience learning, synthetic data pipelines, agentic systems, weak-to-strong generalization, and inference-time scaling.
Learning Resources
Want to get your hands dirty? These open-source repositories and model cards let you run the systems described above.
- TinyRecursiveModels | Samsung SAIL Montréal - GitHub ...official TRM training and evaluation code, with dataset builders for Sudoku, Maze, and ARC-AGI
- tiny-recursive-model | lucidrains - GitHub ...a compact, pip-installable community implementation of TRM
- rlm | Alex L. Zhang - GitHub ...run a Recursive Language Model on your own long documents
- Huginn-0125 | Tom Goldstein's group, University of Maryland - Hugging Face ...set the loop count at inference time with a single num_steps argument
- Ouro-1.4B | ByteDance - Hugging Face ...a looped language model you can load with standard Transformers code
- Introduction to CS and Programming using Python | Dr. Ana Bell - MIT OpenCourseWare ...the full course, including Lecture 16 on recursion over non-numeric data
Additional Viewing
In this full-length MIT 6.100L lecture from Fall 2022, Dr. Ana Bell presents recursion as a divide-and-conquer strategy: break a problem into a base case and a recursive step, then trust the smaller call to do its job. It's the most systematic on-ramp on this page, and it sets up everything in the programming section before you move on to the AI material.
An interview with Alexia Jolicoeur-Martineau, the Samsung SAIL Montréal researcher behind the Tiny Recursive Model, linked from the community TRM implementation. She discusses how TRM simplified HRM into a single two-layer network that refines its own answer in a loop, and why she argues recursion can substitute for raw model size on structured reasoning problems.
An alphaXiv research session from January 6, 2026 with Alex L. Zhang, lead author of Recursive Language Models. The conversation digs into the long-context work: why bigger context windows still suffer from context rot, how loading the prompt into a REPL changes the problem, and when recursive sub-calls earn their extra cost.
Weaviate Podcast #142, in which host Connor Shorten talks with MIT PhD student Alex Zhang about Recursive Language Models as an abstraction for designing agent harnesses. They cover treating prompts as program variables, using recursive sub-agent calls instead of naive context stuffing, speculative programmatic tool calling, running RLMs in the cloud, and what all of this means for search.