Difference between revisions of "Loop"

From
Jump to: navigation, search
m (→‎Feedback Loop - Scientific Discovery)
 
(One intermediate revision by the same user not shown)
Line 308: Line 308:
 
[https://www.youtube.com/results?search_query=~Scientific+Discovery+loop+ai+machine+learning YouTube search...]
 
[https://www.youtube.com/results?search_query=~Scientific+Discovery+loop+ai+machine+learning YouTube search...]
 
[https://www.google.com/search?q=~Scientific+Discovery+loop+ai+machine+learning ...Google search]
 
[https://www.google.com/search?q=~Scientific+Discovery+loop+ai+machine+learning ...Google search]
 +
* [https://www.weforum.org/agenda/2020/11/scientific-discovery-must-be-redefined-quantum-and-ai-can-help/ Scientific discovery must be redefined. [[Quantum]] and AI can help | Dario Gil - World Economic Forum]
 +
* [https://www.the-scientist.com/reading-frames/can-artificial-intelligence-make-scientific-discoveries--65790 Can Artificial Intelligence Make Scientific Discoveries? | Kartik Hosanagar - The Scientist]
  
 
Discovery Loop is a newly formed startup backed by Google and led by former Google Chief Scientist Jeff Dean alongside a team of top AI researchers. The "loop" refers to the scientific method itself: proposing a hypothesis, running an experiment, and evaluating the results. By fully automating this process, the company aims to shift AI from a tool that answers questions into an active, autonomous researcher.  
 
Discovery Loop is a newly formed startup backed by Google and led by former Google Chief Scientist Jeff Dean alongside a team of top AI researchers. The "loop" refers to the scientific method itself: proposing a hypothesis, running an experiment, and evaluating the results. By fully automating this process, the company aims to shift AI from a tool that answers questions into an active, autonomous researcher.  
Line 315: Line 317:
 
* '''Self-Improving Systems:''' The company plans to act as its own first customer. They are starting by using these loops to optimize machine learning research itself. The AI will write code, test it, figure out what went wrong, and write better code. Once the system perfects its own underlying technology, it will expand to tackle global challenges like medicine and clean energy.
 
* '''Self-Improving Systems:''' The company plans to act as its own first customer. They are starting by using these loops to optimize machine learning research itself. The AI will write code, test it, figure out what went wrong, and write better code. Once the system perfects its own underlying technology, it will expand to tackle global challenges like medicine and clean energy.
 
* '''The Team Building It:''' The founders include Jeff Dean and Sanjay Ghemawat, who co-designed the distributed computing backbone of Google, alongside Quoc Le and Oriol Vinyals, pioneers in deep learning and large-scale neural networks.
 
* '''The Team Building It:''' The founders include Jeff Dean and Sanjay Ghemawat, who co-designed the distributed computing backbone of Google, alongside Quoc Le and Oriol Vinyals, pioneers in deep learning and large-scale neural networks.
−
 
−
* [https://www.weforum.org/agenda/2020/11/scientific-discovery-must-be-redefined-quantum-and-ai-can-help/ Scientific discovery must be redefined. [[Quantum]] and AI can help | Dario Gil - World Economic Forum]
 
−
* [https://www.the-scientist.com/reading-frames/can-artificial-intelligence-make-scientific-discoveries--65790 Can Artificial Intelligence Make Scientific Discoveries? | Kartik Hosanagar - The Scientist]
 
−
 
  
 
<hr>
 
<hr>
Line 366: Line 364:
 
Some traders follow the trend, and some go against it. At I Know First we work on algorithmic strategies which are neither, we simply try an assess where the next opportunity is and provide stock market predictions. If this means to do what everyone else does, than why not. If it means going against when everyone else does, this is also fine. The tricky part is determining where this opportunities are, this article will discuss how to find opportunities in what can seem as total randomness. Markets are Complex, but not Unpredictable!  There are two major misconceptions about the stock market. The first one is connected to the classical economic theory which claims markets to be efficient, and as such unpredictable. In this case trying to select one stock over another becomes useless, as no opportunity is ever better than the other. Both stocks are perfectly priced according to their opportunity and risk, with everyone having all information. However, the truth of the matter is that some people profit trading stocks while others lose – this by itself proves the market to be inefficient, and thus exploitable. While US markets are very efficient, and most information is available, not everyone interprets this information the same.  [https://iknowfirst.com/stock-market-predictions-where-in-the-feedback-loop-is-your-portfolio Stock Market Predictions: Where In The Feedback Loop Is Your Portfolio? | I Know First]
 
Some traders follow the trend, and some go against it. At I Know First we work on algorithmic strategies which are neither, we simply try an assess where the next opportunity is and provide stock market predictions. If this means to do what everyone else does, than why not. If it means going against when everyone else does, this is also fine. The tricky part is determining where this opportunities are, this article will discuss how to find opportunities in what can seem as total randomness. Markets are Complex, but not Unpredictable!  There are two major misconceptions about the stock market. The first one is connected to the classical economic theory which claims markets to be efficient, and as such unpredictable. In this case trying to select one stock over another becomes useless, as no opportunity is ever better than the other. Both stocks are perfectly priced according to their opportunity and risk, with everyone having all information. However, the truth of the matter is that some people profit trading stocks while others lose – this by itself proves the market to be inefficient, and thus exploitable. While US markets are very efficient, and most information is available, not everyone interprets this information the same.  [https://iknowfirst.com/stock-market-predictions-where-in-the-feedback-loop-is-your-portfolio Stock Market Predictions: Where In The Feedback Loop Is Your Portfolio? | I Know First]
  
−
 
−
https://iknowfirst.com/wp-content/uploads/2015/06/07.png
 
  
 
{|<!-- T -->
 
{|<!-- T -->

Latest revision as of 11:38, 17 September 2026

YouTube ... Quora ...Google search ...Google News ...Bing News

AI Loops and Agentic Workflows

When developers discuss "loops" in modern AI, they are referring to **Autonomous Agents** running in continuous feedback cycles rather than waiting for step-by-step human prompts.

In traditional generative AI, the interaction is linear: you ask a question, the model generates an answer, and the process stops. In an agentic loop, the AI operates in a continuous cycle—often based on the OODA loop (Observe, Orient, Decide, Act) or similar cognitive architectures.

How an Agentic Loop Works

Instead of just answering a question, an Agent in a loop is given an overarching goal (e.g., "Research the top 5 competitors in this market and create a presentation"). The AI then enters a self-driven cycle:

  1. Observe / Perceive: The Agent looks at the current state of its environment. It might read a webpage, look at an error log, or take a screenshot of your desktop.
  2. Plan / Reason: The Agent breaks the main goal into smaller tasks and decides on the best next step. It might think, "I need to search Google for Competitor A."
  3. Act: The Agent uses an external tool. It might execute a Python script, click a button in a browser, or send an email.
  4. Reflect / Evaluate: This is the critical step that defines a loop. The Agent looks at the result of its action. If the web search failed or the code threw an error, the Agent catches the mistake, revises its plan, and loops back to step 1 to try a different approach.

<GenerateVideo uuid="VIDEO_OVERVIEW.1.1" source="offline"> </GenerateVideo>

Types of AI Loops

  • Single-Agent Loops: Tools like Auto-GPT or OpenClaw use a single LLM to prompt itself repeatedly until a task is finished.
  • Multi-Agent Loops: Frameworks like CrewAI or AutoGen set up multiple specialized Agents (e.g., a "Researcher" and a "Reviewer"). The Researcher finds information and loops it to the Reviewer. If the Reviewer spots an error, it loops the task back to the Researcher, creating a collaborative, self-correcting team.
  • Human-in-the-Loop (HITL): The AI runs its cycle but pauses at critical moments to ask a human for approval before taking a major action, like sending a final email or spending money.

Observe–Orient–Decide–Act (OODA) Loop

YouTube search... ...Google search

The OODA loop is the cycle: Observe–Orient–Decide–Act ...emphasized that "the loop" is actually a set of interacting loops that are to be kept in continuous operation... developed by military strategist and United States Air Force Colonel John Boyd. Boyd applied the concept to the combat operations process, often at the operational level during military campaigns. It is now also often applied to understand commercial operations and learning processes. The approach explains how agility can overcome raw power in dealing with human opponents. It is especially applicable to cyber security and cyberwarfare. Wikipedia ... A Discourse On Winning and Losing | John R. Boyd - Air University Press


We’ve historically focused automation efforts on the “Act” portion but the real potential for new technologies is to address the prior 3 steps: Improving Observation: Improve the data itself with better sensing accuracy, timeliness, relevance, etc or improve our ability to use the data with higher throughput learning processes Improving Orientation: Improve the classification of the current state and the prediction of future states Improving Decision: Improve the ability to choose between paths via better objective functions. How Artificial Intelligence is Closing the Loop with Better Predictions | Erik Trautman - HackerNoon]


Mindset: OODA Loop by John Boyd
OODA Loop is an acronym which stands for Observe, Orient, Decide and Act. In this video I provide a historical, academic, and practical perspective to help shed some light on the topic. John Boyd is one of the greatest military strategist to have lived, and to understand OODA Loop it's import to put it into context. If you have any questions or have something to add, please comment down below. Like this content? Consider supporting me on Patreon where you will get exclusive access to unpublished videos, behind the scenes, coupons, and the satisfaction of supporting pro-freedom content. https://www.patreon.com/WeRunGuns Please LIKE, SHARE, and SUBSCRIBE if you enjoyed!

Balan Ayyar, CEO of Percipient AI, Discusses the OODA Loop Model and Artificial Intelligence
Balan Ayyar is the Founder and CEO of Percipient.ai, a Silicon Valley based artificial intelligence firm focused on delivering products and solutions for the most pressing intelligence and national security challenges. We have been tracking this leader for years and have watched him inspire many to achieve great things. His leadership in senior officer roles in the US Air Force (he is a retired Brigadier General) included tours in combat zones and commands back in the US, including at one point leading the entire Air Force recruiting system. His visionary leadership in the corporate world resulted in the creation of a new firm providing new AI capabilities that are already serving some of the most critical missions in the nation. He is a disciplined visionary with an incredible thurst for learning and a knack for connecting with others. We loved the fact that we recorded him while he was at his desk and when we started asking about what he is reading he could quickly grab his favorite books and give his his insights. It is also always great to talk to a real Air Force leader about the OODA Loop and the model of continuous action and how it applies to not just military ops but business. Topics we covered included: - How the national security establishment needs to change to get ready for the new world - Views on the struggle between open societies and closed/dictatorship/communist societies and how that relationship is changing now. - Role that AI can play to accelerate the return of our economy - Concepts for getting classified information to users where ever they are, including at their homes. Seems technically feasible, and seems like the risk/reward equation may have shifted a bit recently. - Ideas on how US tech firms can better support the nation in the new economy. - Lessons learned from leadership of Percipient that could be helpful to other CEOs or other leaders in government. - Topics that can inform your technology due diligence

Feedback Loop

YouTube search... ...Google search

any process where the outputs of a system are plugged back in and used as iterative inputs. Feedback loops exist just about everywhere. In nature, the evolutionary "arms race" between predators and prey is a classic example. In business, the practice of taking customer feedback (the output of a product or service) and using it to improve future processes is another commonly used feedback loop. Today, rapid advances in artificial intelligence (AI) and machine learning are helping businesses do more with data. These systems — and their ability to analyze an inhuman amount of data — allow businesses to adjust algorithms, workflows and processes on the fly. Get More Out Of Feedback Loops With AI | Arka Dhar - Forbes


Feedback loops: How nature gets its rhythms - Anje-Margriet Neutel
While feedback loops are a bummer at band practice, they are essential in nature. What does nature’s feedback look like, and how does it build the resilience of our world? Anje-Margriet Neutel describes some common positive and negative feedback loops, examining how an ecosystem’s many loops come together to make its ‘trademark sound.’ Lesson by Anje-Margriet Neutel, animation by Brad Purnell.

How Games Use Feedback Loops | Game Maker’s Toolkit
Playing Pyre over Christmas got me thinking about feedback loops: the reward structures in games that can reinforce or balance out winners and losers. In this episode I’ll explain what this all means, and talk about the design of Pyre’s positive and negative loops.

Multi-Loop Learning

YouTube search... ...Google search



SINGLE & DOUBLE LOOP LEARNING
Made by: Hannah Munshi 2016

Managing for Quality Lecture Series - #6 Organizational Learning – Triple Loop Experience
American Society for Quality, Quality Management Division With Dr. Gregory Watson The concept of “Triple-Loop Learning” grew out of the work of Harvard psychologist Chris Argyris and Donald Schön as a “reflective management practice.” The first loop occurs as the outcome of work is examined when is nonconforming to its performance expectations. This cycle of learns about problem-solving, standardization, and process control. The second learning loop occurs reviews the methods used in the first loop to improve efficiency and effectiveness of work management methods. The third learning loop occurs as management examines its business assumptions for driving operational excellence. This webinar describes this process and illustrates it in a case study.

François Chollet: Limits of Deep Learning | AI Podcast Clips
Lex Fridman from Sep 2019. François Chollet is the creator of Keras, which is an open source deep learning library that is designed to enable fast, user-friendly experimentation with deep neural networks. It serves as an interface to several deep learning libraries, most popular of which is TensorFlow, and it was integrated into TensorFlow main codebase a while back. Aside from creating an exceptionally useful and popular library, François is also a world-class AI researcher and software engineer at Google, and is definitely an outspoken, if not controversial, personality in the AI world, especially in the realm of ideas around the future of artificial intelligence.

Understanding Triple Loop Learning and it's impact on the Coaching Process
The results that people achieve in life come from the actions that they take both at conscious and unconscious levels. Often a client will express a desire for change, and seek actionable steps from his coach to create that change. Taking positive actions can lead to positive changes and results, but may still fall short of being transformational. Coaches can help clients achieve real breakthrough when they understand how Triple Loop Learning occurs. For more, please visit us at https://www.coachmastersacademy.com/

How to use double loop learning
pfc Social Impact Advisors

Double-loop learning: a case study from the front-line | Roderic Yapp | TEDxWandsworth
Roderic Yapp is a former Royal Marines Officer, he led marines on operations around the world including the front-line in Afghanistan in 2007, and evacuated civilians from Libya during the Arab Spring. Roderic delves into his experience as a Royal Marine explaining why we must challenge our own behaviour. He uses examples of changes made by UK Military and the Taliban to explain why we must challenge how we think if we are to solve tomorrow's problems effectively. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at https://ted.com/tedx

Triple Golden OODA

YouTube search... ...Google search

The Triple Golden OODA loop diagram depicts a hybrid of concepts from the Triple Loop concept inspired by Chris Argyris & Donald Schön's work on the Double Loop, Simon Sinek's Golden Circle in 'Start with Why', and Colonel John Boyd's Observe, Orient, Decide and Act (OODA) Loop from his ‘A Discourse On Winning and Losing'.

TripleGoldenOODA.png

Start with WHY - Simon Sinek In Vegas
IDLife NYC Those who lead inspire us. We follow those who lead because we want to and for selfish reasons. Those who start with "why" that have the ability to inspire those around them or find others who inspire them.

1. You have to know WHY you do what you do.

2. You have to have DISCIPLINE of how.

3. You have to have consistency of WHAT.

Lucy and the Chocolate Factory
CBS All Access -- How best can Triple Golden OODA loop be applied? What would be the Single loop in this situation? The Double loop? The Triple loop? The Why?

Feedback Loop - Peer Learning

YouTube search... ...Google search

5f5a0d4872a99ec9cde81526_engagement.gif

Google AI's Take on How To Fix Peer Review
The paper "Avoiding a Tragedy of the Commons in the Peer Review Process" is available here: https://arxiv.org/abs/1901.06246

Scientific publications and scientific peer review - Can AI be a solution to current Issues
Dan L. Clinciu a researcher with CMU, and an assistant professor with NCUT, both in Taichung Taiwan and Brian D. Oscar discuss various issues in current scientific peer review and publications and how AI might be a solution to these various issues in the near future. They also discuss how significant scientific discoveries could have disastrous outcomes if they are misinterpreted. www.het-forum.net

Feedback Loop - Creating Consciousness

YouTube search... ...Google search


Michio Kaku: Feedback loops are creating Consciousness | Big Think
One of the great questions in all of science is where consciousness comes from. When it comes to consciousness, Kaku believes different species have different levels of consciousness, based on their feedback loops needed to survive in space, society, and time. According to the theoretical physicist, human beings' ability to use past experiences, memories, to predict the future makes us distinct among animals — and even robots (they're currently unable to understand, or operate within, a social hierarchy). Dr. Michio Kaku is the co-founder of string field theory, and is one of the most widely recognized scientists in the world today. He has written 4 New York Times Best Sellers, is the science correspondent for CBS This Morning and has hosted numerous science specials for BBC-TV, the Discovery/Science Channel. His radio show broadcasts to 100 radio stations every week. Dr. Kaku holds the Henry Semat Chair and Professorship in theoretical physics at the City College of New York (CUNY), where he has taught for over 25 years. He has also been a visiting professor at the Institute for Advanced Study at Princeton, as well as New York University (NYU).

Investigating Consciousness with Closed-Loop Neural Reinforcement
Neuroscience Virtual Event 2018 Aurelio Cortese, PhD Researcher, ATR Institute International Aurelio received his MSc degree from the Life Sciences and Technologies faculty at EPFL in Switzerland (2012), while spending one year at the Max-Planck Institute of Psychiatry in Munich working on mice models of Alzheimer's disease and depression. He is now a researcher at ATR working with Mitsuo Kawato, with ongoing collaborations with Hakwan Lau's Lab at UCLA and Hong Kong University, and with Benedetto de Martino at UCL/Wellcome Trust (London, UK). Webinar: Investigating Consciousness with Closed-Loop Neural Reinforcement Abstract: In consciousness studies, a longstanding controversy concerns whether activity in the prefrontal cortical (PFC) region of the brain is necessary to evoke conscious experiences. Similarly, there is contrasting evidence on whether subjective confidence directly reflects sensory evidence or may depend on a late-stage estimation, related to consciousness but dissociable from sensory processes. As of yet, in humans, experimental tools have lacked the power to resolve these issues convincingly. We overcome this difficulty by capitalizing on the recently developed method of decoded neurofeedback (DecNef), where the occurrence of distinct neural events (e.g., spatial activation patterns) is selectively rewarded. This closed-loop training thus has the power of reinforcing purely content-specific processes that typically lie below consciousness. In a series of recent studies, we employed DecNef to directly reinforce neural activation patterns in areas related to representations at different levels of complexity, from simpler (e.g. orientation in visual cortex) to more composite (e.g. confidence in PFC). In all cases, the manipulations resulted in clear behavioral or physiological changes. Nevertheless, during the training sessions, participants were never conscious about the content of these localized recurring activation patterns. This raises a very interesting point: it is likely that consciousness requires more than just a local representation, however well defined. As already proposed, consciousness may rely on concomitant activations across frontoparietal networks. Frontal cortices could thus play a crucial role in bringing content to consciousness, by virtue of being also implicated in higher order representations. Approaches combining machine learning techniques with brain imaging and closed-loop training such as DecNef may offer a strong paradigm to further explore and understand consciousness and its real neural basis.

MIT Godel Escher Bach Lecture 1
jasonofthel33t

Introduction to Brain and Consciousness 2.2 - Feedforward and Feedback Connections
Synergos Narrated by Julia Donnell Written by Hakwan Lau Now, onto more details about the pathways. Again, getting the basic concepts is much more important than remembering all the specifics. When we say there’s a pathway, we imply there is a direction of information flow. And in fact there is. From the eyeball towards the brain, that is how perception happens. We call this direction feedforward, or bottom up. But interestingly, information also flows from the other direction. And we call these feedback, or top down signals. Directed/Edited by Justine Suh Cinematography/Sound Editing by Emily Kalish Key Grip and Sound by Santiago Rodriguez Herran Graphics & Subtitles by Peter Keating & Charmaine Wong FAIR USE NOTICE: This video contains copyrighted material the use of which has not always been specifically authorized by the copyright owner.

Feedback Loop - Scientific Discovery

YouTube search... ...Google search

Discovery Loop is a newly formed startup backed by Google and led by former Google Chief Scientist Jeff Dean alongside a team of top AI researchers. The "loop" refers to the scientific method itself: proposing a hypothesis, running an experiment, and evaluating the results. By fully automating this process, the company aims to shift AI from a tool that answers questions into an active, autonomous researcher.

  • Automating the Scientific Method: Traditionally, human teams test ideas sequentially. Discovery Loop is building systems that let AI handle the entire propose-run-evaluate cycle. Think of it like a self-driving car, but for scientific research. Instead of a scientist manually steering the experiment, the AI navigates the data, adjusts its route based on roadblocks, and drives toward a solution.
  • Extreme Parallelism: Because the loop is automated, the system can run thousands of experiments simultaneously. If finding a new battery material is like searching a massive maze, human researchers walk down one path at a time. An automated loop drops thousands of explorers into the maze at once, drastically shrinking the time it takes to find the exit.
  • Self-Improving Systems: The company plans to act as its own first customer. They are starting by using these loops to optimize machine learning research itself. The AI will write code, test it, figure out what went wrong, and write better code. Once the system perfects its own underlying technology, it will expand to tackle global challenges like medicine and clean energy.
  • The Team Building It: The founders include Jeff Dean and Sanjay Ghemawat, who co-designed the distributed computing backbone of Google, alongside Quoc Le and Oriol Vinyals, pioneers in deep learning and large-scale neural networks.

The paradigm shift will be from AI used for analysing the data which has already been obtained, to AI deciding what to measure next. Why A‍r‍tificial Intellig‌‍enc‍e Will Ena‍ble New Sc‌ientific‍ D‍isc‍over‌‍ies - Andrew Briggs - Graphcore


[Quantum Computing and AI to Enable Our Sustainable Future | Katia Moskvitch - ]IBM


Toward Robot Scientist for autonomous scientific discovery | A. Sparkes, W. Aubrey, E. Byrne, and A. Clare - ResearchGate

Machine Learning Accelerating Scientific Discovery
Phil Nelson, Google Research

Hypothesis Generation with AGATHA : Accelerate Scientific Discovery with Deep Learning | AISC
ML Explained - Aggregate Intellect - AISC For slides and more information on the paper, visit https://ai.science/e/hypothesis-generation-with-agatha-accelerate-scientific-discovery-with-deep-learning--2020-04-01 Discussion lead: Justin Sybrandt Discussion facilitator(s): Rouzbeh Afrasiabi Medical research is risky and expensive. Drug discovery, as an example, requires that researchers efficiently winnow thousands of potential targets to a small candidate set for more thorough evaluation. However, research groups spend significant time and money to perform the experiments necessary to determine this candidate set long before seeing intermediate results. Hypothesis generation systems address this challenge by mining the wealth of publicly available scientific information to predict plausible research directions. We present AGATHA, a deep-learning hypothesis generation system that can introduce data-driven insights earlier in the discovery process. Through a learned ranking criteria, this system quickly prioritizes plausible term-pairs among entity sets, allowing us to recommend new research directions. We massively validate our system with a temporal holdout wherein we predict connections first introduced after 2015 using data published beforehand. We additionally explore biomedical sub-domains, and demonstrate AGATHA's predictive capacity across the twenty most popular relationship types. This system achieves best-in-class performance on an established benchmark, and demonstrates high recommendation scores across subdomains. Reproducibility: All code, experimental data, and pre-trained models are available online:

Feedback Loop - Stock Market Predictions

YouTube search... ...Google search

Why you can beat the market, even when it does not seem so. The importance of loops, patterns, and predictable events. Random events are don’t measure risks, and should not affect your decision making. Some traders follow the trend, and some go against it. At I Know First we work on algorithmic strategies which are neither, we simply try an assess where the next opportunity is and provide stock market predictions. If this means to do what everyone else does, than why not. If it means going against when everyone else does, this is also fine. The tricky part is determining where this opportunities are, this article will discuss how to find opportunities in what can seem as total randomness. Markets are Complex, but not Unpredictable! There are two major misconceptions about the stock market. The first one is connected to the classical economic theory which claims markets to be efficient, and as such unpredictable. In this case trying to select one stock over another becomes useless, as no opportunity is ever better than the other. Both stocks are perfectly priced according to their opportunity and risk, with everyone having all information. However, the truth of the matter is that some people profit trading stocks while others lose – this by itself proves the market to be inefficient, and thus exploitable. While US markets are very efficient, and most information is available, not everyone interprets this information the same. Stock Market Predictions: Where In The Feedback Loop Is Your Portfolio? | I Know First


How Do Stock Trading Algorithms Work?
The stock market can be a foracious beast to those that don't understand it, but nowadays, you don't even need to understand it to make money. The rise of the digital information age and AI has brought about a new way of stock trading called algorithmic trading. Sometimes referred to as automated trading or black-box trading, this is essentially a program that can trade stocks at high speeds and frequencies perfectly in line with the market. These programs are given constraints and instructions like timing, price, amount, etc. and a user can fine tune how they exactly work. So how does this all work then... let's take a look. All images courtesy of Creative Commons or protected under Fair Use. For questions or concerns about the use of any media, please contact the page directly.

Feedback Loop - The AI Economist

YouTube search... ...Google search

Optimal Tax Design as Learned Reward Design Using Reinforcement Learning. Reinforcement learning is a powerful framework in which Agents learn from experience collected through trial-and-error. We use model-free RL, in which Agents do not use any prior world knowledge or modeling assumptions. Another benefit of RL is that Agents can optimize for any objective. In our setting, this means that a tax policy can be learned that optimizes any social objective, and without knowledge of workers’ utility functions or skills. The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies | S. Zheng, A. Trott, S. Srinivasa, N. Naik, M. Gruesbeck, D. Parkes, and R. Socher - Einstein.ai]

AI Economist: How can AI FIX taxes? (Deep Reinforcement Learning, Machine Learning, Economics)
Review and in depth analysis of very interesting paper out of Harvard and Salesforce Research.

OUTLINE:

0:10 - Economic background, equality vs productivity tradeoff, tax schedule 2:18 - Deep Reinforcement Learning Network Overview 3:45 - Multi-Agent Markov Games 4:22 - Figure 1: Game map, explanation 5:21 - Results, No taxes (division of labor) 6:58 - Notation table 7:56 - Objective, individual taxpayer 9:22 - Defining utility function 11:22 - Taxes Tax Brackets, Social Welfare Functions 12:50 - Objective for policy planner 14:00 - Two-level reinforcement learning training scheme 15:26 - U.S. Policy vs AI Economist vs Saez

Paper: https://arxiv.org/abs/2004.13332

Feedback Loop - Synthetic

YouTube search... ...Google search

The Wandering Dreamer: An Synthetic Feedback Loop
This experiment uses four machine learning models to create a feedback loop between synthesized images and text. All of the images you see here are fabricated, as is the text that describes each image. Made by Brannon Dorsey using Runway. Source code

1. The first row of images are produced from a class label using BigGAN. 2. The text below is an auto-generated caption of the BigGAN image using Im2Text. 3. The next set of images are synthesized by an Attentional GAN using the auto-generated captions. 4. The text at the bottom classifies the image above it using MobileNet. This class label is then sent back to BigGAN as input to create an infinite loop.

Feedback Loop - Allostasis & Homeostasis

  • Allostasis the process of maintaining stability by changing the body’s operating settings in response to context and predicted demand—your brain and body anticipate needs (stress, exercise, threat, illness, time of day) and adjust hormones, autonomic output, and immune activity to match the situation.

Before a speech: your heart rate rises, cortisol may increase, and blood pressure shifts before you start talking → Allostasis

  • Homeostasis is the state of steady internal, physical, and chemical conditions maintained by living systems. This is the condition of optimal functioning for the organism and includes many variables, such as body temperature and fluid balance, being kept within certain pre-set limits (homeostatic range). Homeostasis operates primarily through negative feedback loops. When a variable changes, the body triggers a response to reverse that change to maintain equilibrium.

After the speech ends: those values return toward baseline through feedback corrections → Homeostasis


Allostasis as the strategy (adapt to demand) and Homeostasis as the fine-tuning mechanics (keep variables in safe ranges).

How they’re different

Goal
Homeostasis: keep a variable near a set range.
Allostasis: keep the organism functioning well by adapting the set range and related systems to the moment.
Timing
Homeostasis: mostly reactive (corrects after deviation).
Allostasis: often predictive/anticipatory (adjusts before or during expected demand).
Control style
Homeostasis: local “thermostat” loops (negative feedback).
Allostasis: coordinated, whole-system regulation (brain-driven orchestration across multiple systems).
Cost concept
Homeostasis: not usually framed as “costly” unless failing.
Allostasis: emphasizes wear-and-tear from chronic activation—allostatic load.

Homeostatic Control System A homeostatic control system consists of three functional components:

  1. Receptor (Sensor): Detects a change in the environment (stimulus).
  2. Control Center: Processes the information and signals the effector (e.g., the brain).
  3. Effector: Carries out the necessary adjustment to restore balance.

The term was coined by physiologist Walter Bradford Cannon in 1926. It is derived from the Greek words:

Homeostasis & Allostasis: An Evolutionary Perspective With Consciousness

Life survives by keeping its internal conditions in a workable range. Early life mostly did this with Homeostasis (reactive feedback). As nervous systems evolved, organisms gained Allostasis (predictive, context-sensitive regulation). What we call conscious experience may be one of the tools that helps organisms coordinate allostatic control across the whole body—especially when tradeoffs, uncertainty, or social complexity are involved.

Homeostasis came first: the ancient “stay alive” machinery Homeostasis is evolution’s baseline: keep core variables (ions, pH, temperature, glucose, oxygen, hydration) within survivable bounds.

  • Single cells: membrane pumps, ion channels, osmotic regulation, energy balance (ATP) — fast local loops.
  • Multicellular life: internal transport (circulation), compartment control, hormonal signaling — keeping the “internal ocean” stable enough for specialized tissues.

Key evolutionary point: Homeostasis works best when the environment is fairly predictable and when local feedback loops can correct deviations quickly.

Allostasis emerged with nervous systems: “stability through change” Allostasis is what you get when an organism can anticipate, learn, and coordinate multiple systems at once.

  • Prediction: adjust physiology before demand hits (fight/flight, exercise, cold exposure, feeding).
  • Coordination: align heart, lungs, immune tone, attention, and behavior toward the same goal.
  • Tradeoffs: choose a “good enough” body state for the situation (e.g., temporarily raise blood pressure to escape, even if it’s not ideal long-term).

Key evolutionary point: Allostasis becomes valuable when:

  • the environment is variable,
  • the organism is mobile,
  • the organism must make fast choices under uncertainty.
  • and playing dead

Where consciousness fits: interoception + choice under uncertainty A useful way to connect consciousness to regulation is interoception — sensing internal body states (heartbeat, breathing, gut signals, temperature, inflammation, hormonal state).

  • Much interoceptive control is unconscious (reflex loops in the brainstem/hypothalamus).
  • Some interoceptive information becomes conscious feelings (hunger, thirst, nausea, calm, anxiety, “wired/tired,” pain).

One evolutionary hypothesis: consciousness helps Allostasis when the organism needs flexible control across time:

  • planning (“If I keep running, will I overheat?”)
  • learning (“That food made me sick—avoid it.”)
  • social prediction (“If I challenge him, what happens next?”)
  • conflict resolution (“I’m scared, but I still need to do this.”)

A layered control stack: from reflex → feeling → deliberation You can think of regulation as a stack of increasingly “expensive” control systems:

Layer A
Local Homeostasis (fast, automatic)
Cells and organs self-regulate (ion balance, local blood flow, basic reflexes).
Layer B
Body-wide Allostasis (coordinated, mostly automatic)
Brainstem + hypothalamus + autonomic and endocrine systems shift setpoints based on context (stress response, circadian rhythm, fever response).
Layer C
Conscious control (slow, flexible, meaning-driven)
Cortex can simulate futures, apply goals/values, and reshape behavior to prevent future instability (choose shelter, negotiate, delay gratification, seek help).

Why it matters: consciousness is not required for most homeostatic corrections—but it may be crucial for complex allostatic problems where the “right” action depends on goals, predictions, and social context.

Emotions as “Allostatic programs” From this lens, emotions aren’t random “extras.” They can be seen as packaged action-and-body-state programs:

  • Fear biases attention + raises arousal to escape threats.
  • Anger mobilizes energy for confrontation and boundary enforcement.
  • Sadness can promote withdrawal/conservation and social support seeking.
  • Joy/interest can promote exploration and learning when safe.

These states are Allostatic because they change physiology and behavior together to manage risk and opportunity.

When the system misfires: Allostatic load Allostasis is powerful, but it has a cost when it’s chronically engaged:

  • persistent stress physiology
  • disrupted sleep/circadian rhythms
  • metabolic strain
  • inflammatory changes

Allostatic load is the “wear-and-tear” that shows up when the body keeps paying short-term survival costs without recovery—often a mismatch between ancient regulatory systems and modern persistent stressors.

Takeaway

  • Homeostasis is the ancient reactive core: correct deviations to protect life.
  • Allostasis is the evolved upgrade: predict, coordinate, and adapt setpoints to meet situations.
  • Consciousness may be one of evolution’s tools for higher-level Allostatic control—helping an organism integrate internal signals with goals, learning, and social reality when simple reflexes aren’t enough.

Examples

Process Scenario Homeostatic Response
Thermoregulation Hyperthermia (Too hot) Sweating (evaporative cooling) and vasodilation (widening of blood vessels to release heat).
Hypothermia (Too cold) Shivering (muscle heat generation) and vasoconstriction (narrowing of blood vessels to conserve heat).
Glucoregulation Hyperglycemia (High blood sugar) The pancreas releases insulin, causing cells to absorb glucose.
Hypoglycemia (Low blood sugar) The pancreas releases glucagon, causing the liver to release stored glucose.
Osmoregulation Dehydration The kidneys concentrate urine to conserve water; the hypothalamus stimulates thirst.

Vagus Nerve

Vagus nerve acts like a two-way control cable between your organs and your brainstem—constantly measuring internal conditions and then nudging things back toward “normal.” Most vagus fibers are sensory (afferent)—they carry status updates from your organs up to the brain (often cited around ~80% afferent). Those signals primarily land in the nucleus tractus solitarius (NTS) in the medulla, a major “autonomic dashboard” that integrates input from the cardiovascular, respiratory, and digestive systems. Then the brain sends motor (efferent) vagal output back down (via nuclei like the nucleus ambiguus and dorsal motor nucleus) to adjust organs—usually in a calming, “rest-and-digest” direction.

Organ state changes → vagal sensory input → NTS integrates → vagal output → organ function shifts → state normalizes

  1. Blood pressure & heart-rate control (baroreflex) - Sensors in major arteries detect stretch/pressure and signal the brainstem; the NTS then helps drive vagal output that slows the heart when pressure is high (and relaxes vagal braking when pressure is low).
  1. Breathing–heart coordination (respiratory sinus arrhythmia) - Your heart rate naturally speeds a bit on inhale and slows on exhale; that rhythm reflects tight coupling between respiration networks and cardiac vagal control pathways.
  1. Gut “status,” satiety, and digestion (vagovagal reflexes) - Vagal sensory neurons detect things like stomach stretch and digestive signals and report to the brainstem; vagal output helps tune motility and secretion.
  1. The “inflammatory reflex” (immune homeostasis) - There’s evidence for a vagus-linked reflex where vagal signaling can dampen excessive cytokine release (often described as the cholinergic anti-inflammatory pathway), involving acetylcholine signaling and α7 nicotinic receptors on immune cells in experimental models


AlphaFold 2

DeepMind AlphaFold can help researchers understand receptors/ion channels/drug targets that sit upstream of autonomic signaling. (think adrenergic + muscarinic GPCRs, serotonin receptors, nicotinic receptors, cardiac ion channels like hERG/Kv11.1, etc.).

  • 3D structure when no experimental model exists: DeepMind AlphaFold can generate a workable 3D fold for hard targets (especially membrane proteins), giving researchers a structural starting point for autonomic-relevant receptors and channels.
  • Druggable pocket mapping: With a predicted structure, researchers can identify and compare likely orthosteric/allosteric pockets and map known functional residues to guide selectivity and SAR hypotheses.
  • Signaling-complex hypotheses: DeepMind AlphaFold (including multimer approaches) can model receptor–transducer interfaces (e.g., GPCR–G protein) to propose coupling-selectivity determinants and testable interaction residues.
  • Complexes with ligands/ions (AlphaFold 3): DeepMind AlphaFold 3 can propose bound geometries for proteins with small molecules, ions, and modified residues, helping frame how upstream autonomic targets may engage drugs or cofactors.
  • Channel gating and state-dependent binding: Structural models can suggest pore architecture and gating features that help explain why some drugs bind preferentially to specific channel states relevant to excitability control.
  • Variant-to-mechanism interpretation: By placing mutations on a structure, researchers can hypothesize whether variants affect folding/trafficking, ligand binding, gating, or signaling interfaces and then design focused validation assays.
  • Faster experimental design and interpretation: DeepMind AlphaFold models help pick construct boundaries and stabilizing changes and can speed cryo-EM/mutagenesis planning by providing a structural scaffold for iterative testing.

Active Inference Loop

YouTube search... ...Google search

Active inference is a framework that treats intelligent systems like scientists instead of passive pattern-matching engines. Most modern machine learning models operate as giant autocomplete algorithms. They compress massive datasets to guess outputs, but if a scene deviates from their training data, they don't adjust well. Their confidence stays high even when their performance drops. Active inference takes a different approach. It unifies perception, learning, and action into a single continuous loop. Instead of just reacting to the world, an Agent anticipates what will happen next and actively updates its internal model when reality doesn't match its predictions.

This framework gives an Agent an explicit generative model that interacts with the outside world through a sensory and action boundary known as a Markov blanket. The Free Energy Principle and its role in Active Inference represent a shift in how we understand intelligence, moving from a system that passively reacts to stimuli to one that actively anticipates and structures reality.

The Free Energy Principle At its core, the free energy principle suggests that intelligent systems, whether biological or artificial, behave as if they are constantly trying to minimize Variational Free Energy.

  • What it measures: It acts as a score of the mismatch between the system's internal model of the world and its actual sensory observations, while also penalizing overly complex internal models.
  • The Goal: By minimizing this "surprise," the system maintains stability and keeps itself in a predictable, safe state. If a system is constantly shocked by its environment, it signifies that it lacks understanding or control over its circumstances.

Active Inference Framework Active inference extends this by unifying perception, learning, and action into a single loop. It uses the free energy principle to guide how an Agents interacts with the world:

  • Perception as Inference: Instead of recording the world passively, the brain (or AI Agent) maintains hypotheses about "hidden causes" it cannot directly see. It constantly updates these beliefs based on sensory evidence.
  • Action as Sampling: An Agent doesn't just infer the world; it actively chooses actions that help it "sample" the world to resolve uncertainty. For example, if you are unsure about an object, you might change your perspective to get a better view rather than just relying on the initial, noisy signal.
  • Solving the Exploration-Exploitation Dilemma: Unlike traditional AI that relies on hand-designed reward signals, Active Inference replaces this with the minimization of expected free energy. This naturally drives an Agent to explore the environment to gain information (reduce uncertainty) while also exploiting what it knows to achieve preferred outcomes.

Essentially, intelligence is reframed as self-evidencing. It is a continuous process of acting to confirm and refine one's understanding of the world while avoiding states that would overwhelm the system with unpredictability. Ultimately, integrating these structured mechanics with large language models offers a practical path toward efficient, explainable AI that explicitly understands the limits of its own knowledge.

This physics idea might be the next generation of machine learning
CompuFlair explains how Active Inference uses physics principles to address the limitations of modern machine learning by treating AI as a system that actively anticipates reality rather than passively reacting to it. Current models often struggle with unpredictable environments because they operate as pattern-matching black boxes without an internal logic of cause and effect. To solve this, Active Inference gives an Agent an explicit generative model, which interacts with the outside world only through a sensory and action boundary known as a Markov blanket. Rather than chasing arbitrary rewards, the system is driven by a single optimization principle: minimizing variational free energy to reduce the mismatch between its predictions and incoming sensory data. This continuous loop naturally resolves the exploration-exploitation dilemma, as the Agent actively samples its environment to resolve uncertainty and manage precision.

Markov Blanket

YouTube search... ...Google search

To understand how an AI Agent or a biological organism interacts with the world, you have to define where the system ends and the environment begins. This boundary is called a Markov blanket. It is a mathematical concept that acts as a statistical shield. It separates the internal states of a system from the external states of the universe.

Because of this boundary, the internal workings of a system are conditionally independent of the outside world. Everything the system needs to know about the environment is contained entirely within the blanket itself. The Markov blanket consists of two crucial parts:

  • Sensory states: These take in information from the outside world but are not influenced by the system's internal states.
  • Active states: These push influence out into the world but are not directly altered by external forces.

By maintaining this boundary, an intelligent system can sample the world through its sensors and act upon it, all while keeping its internal model intact. You never access reality directly. Instead, you constantly update your beliefs based on the signals crossing your Markov blanket. This concept forms the foundation of how systems self-organize, survive, and make sense of their surroundings without simply dissolving into chaos.

Markov blankets and Bayesian mechanics (Karl Friston)
Karl Friston explores the mathematics of consciousness and self-organization. He explains how predictive processing flips the traditional view of perception. Instead of passively receiving data, systems actively construct explanations from the inside out. He breaks down how a Markov blanket provides the necessary boundary for an Agent to maintain its internal states while using sensory and active states to navigate the world.

Me and My Markov Blanket Sentience and the Free Energy Principle | Lecture
In this Science Gallery Bengaluru lecture, Karl Friston discusses sentience as a form of inference. He breaks down the physics of self-organization, showing how any system needs a Markov blanket to distinguish itself from its environment. The talk highlights how sensory and active states act as an interface, allowing biological and artificial systems to minimize surprise and maintain their structure in an unpredictable universe.

Recursion

YouTube search... ...Google search

Tower_of_Hanoi_4.gif

Recursion for Beginners: A Beginner's Guide to Recursion
Recursion has an intimidating reputation for being the advanced skill of coding sorcerers. But in this tutorial we look behind the curtain of this formidable technique to discover the simple ideas under it. Through live coding demos in the interactive shell, we'll answer the following questions:

  • What is recursion, and when is it a good idea and bad idea to use it?
  • What's a stack, the call stack, and a stack overflow?
  • What are all the confusing ways that recursion is commonly taught?
  • Do some problems require recursion? Can recursion do anything a loop can't?
  • What is memoization, and how does functools.lru_cache work?
  • How do I draw
Retrieved from "https://primo.ai/index.php?title=Loop&oldid=44339"