Architectures

From PRIMO.ai
Jump to navigation Jump to search


YouTube ... Quora ...Google search ...Google News ...Bing News

Additional architecture topics:

Page map: Recent Developments · Technical Architecture · Technologies Implemented · Decision Models · Ecosystem Fit · Additional Viewing · Sources and Further Reading

Looking for sources? Annotated reading lists by topic are in Sources and Further Reading. Sources for each decision-model offering (Jev, Clef, pplx-decider, and the rest) are listed under that offering in the Decision Models part.


An AI architecture describes how the parts of an artificial intelligence system are organized and how information moves between them. When people say "architecture," they can mean any of five different levels, and it helps to know which one is on the table:

  • Model architecture: the structure of the network itself, such as a transformer, convolutional network, graph neural network, mixture-of-experts model, state-space hybrid, looped model, or Joint Embedding Predictive Architecture. See Neural Architecture.
  • Learning architecture: how the model learns, including supervised, self-supervised, reinforcement, generative, predictive, and transfer-learning approaches. Since 2025, this level also includes the reinforcement-learning stage that turns a base model into a "reasoning" model, and the choice of optimizer that keeps giant training runs stable.
  • System architecture: how models connect with memory, retrieval, tools, databases, applications, APIs, verifiers, and other services.
  • Compute architecture: the CPUs, GPUs, TPUs, accelerators, networking, storage, and distributed infrastructure used to train and run the system.
  • Deployment architecture: how the finished system runs across cloud, edge, mobile, robotic, enterprise, or decentralized environments.

A few research lineages define today's landscape:

  • Transformers: Ashish Vaswani and colleagues at Google introduced the transformer in 2017.
  • Mixture-of-experts: That same year, Noam Shazeer and colleagues revived a much older idea (the 1991 mixture-of-experts work of Robert Jacobs, Michael Jordan, Steven Nowlan, and Geoffrey Hinton) as a sparsely gated layer for giant networks.
  • State-space models: Albert Gu and Tri Dao pushed state-space models into the mainstream with Mamba in 2023. Their team returned in 2026 with Mamba-3, an ICLR oral paper.
  • JEPA: Yann LeCun laid out the Joint Embedding Predictive Architecture (JEPA) in a 2022 position paper and has spent the years since arguing that world models, not bigger text predictors, are the path to machine intelligence.
  • Spatial intelligence: Fei-Fei Li's World Labs made the opposite bet, building generative world models. AMD agreed to buy the company for $8.2 billion in September 2026.
  • Reasoning through reinforcement learning: DeepSeek showed in 2025 that reinforcement learning alone could teach a large model to reason, in the first major LLM paper to pass peer review at Nature.
  • System One models: In September 2026, Diogo Almeida's TypeSafe AI introduced "System One" models that give up text generation entirely to return typed, calibrated decisions. Within about two weeks, OpenAI, Cloudflare, Perplexity, and AWS had shipped decision models of their own (see Decision Models).

The big shift of the past two years is that architecture stopped meaning "pick one model." Production AI now combines several specialized parts, each doing a different job:

  • Sparse experts for capacity.
  • Gated, sparse, and hybrid sequence layers for long context.
  • A reinforcement-learning stage for reasoning.
  • Retrieval and tools for knowledge and action.
  • Verifiers and routers for reliability.
  • Increasingly, world models for physical tasks.

A single system might pair a transformer core with a mixture-of-experts layer for scale, vector or graph storage for memory, external tools for actions, and a fast decision model as a gatekeeper, all running on GPUs and deployed as containers or serverless functions.


The question isn't "Which AI architecture is best?" The question is "Which combination of architectures best fits the problem you're solving, at the cost, latency, and reliability you can afford?"


  • The Mechanism: Every AI architecture answers four design questions:
    1. How do raw inputs become internal representations (embeddings)?
    2. How does information mix across positions or entities: through attention, recurrent or state-space updates, convolution, or graph message passing?
    3. How is computation routed? It can flow through every parameter (dense), through a few selected experts (sparse Mixture-of-Experts (MoE)), through the same block looped several times, or through a cascade of cheap and expensive models.
    4. What is the model trained to predict: the next token, a masked latent representation, a denoised sequence, a verifiably correct answer, or a typed decision with a probability attached?
A fifth choice used to be treated as a training detail, but it now shapes architecture too: which optimizer trains the network, and how it keeps attention logits from exploding. Small changes can matter enormously. A single sigmoid gate after attention, or a clip on query and key weights, changes where attention mass goes and whether a trillion-parameter run finishes at all. Change any one of those answers and you change what the system is good at, how much context it can use, and how well it supports In-Context Learning (ICL).
  • The Physics: Architectures live or die on hardware constraints. Standard softmax attention does work that grows with the square of sequence length, and its key-value (KV) cache grows linearly with context.
    • Worked example: The cache costs roughly 2 × layers × KV heads × head dimension × bytes per value for every token. A hypothetical model with 60 layers, 8 KV heads, 128-dimensional heads, and 16-bit values needs about 0.25 MB per token, or roughly 32 GB for a single 128,000-token conversation, before counting the weights.
    • Memory bandwidth: Token-by-token decoding is usually limited by memory bandwidth rather than arithmetic. The chip spends most of its time moving weights and cache, not multiplying.
    • Sparse attention: DeepSeek's sparse attention attacks the context cost head-on by attending to only the 2,048 most relevant earlier tokens. That cut long-context serving prices by roughly six to seven times.
    • Sparse experts: Mixture-of-experts cuts arithmetic per token, but every expert still has to sit in memory, and routing adds all-to-all communication between accelerators.
    • Fixed-size state: Linear-attention and state-space layers replace the growing cache with a fixed-size state.
    • Parallel decoding: Diffusion language models and Jev's parallel sampler raise arithmetic intensity, so GPUs stay busy instead of waiting on memory.
    • Event-driven hardware: Neuromorphic chips attack the same energy problem from the other side, putting memory next to compute and only doing work when events arrive.
See Processing Units - CPU, GPU, APU, TPU, VPU, FPGA, QPU.
  • Why This Matters: Architecture sets your cost per decision, your latency floor, your reliability ceiling, and which problems are even reachable. The stakes are concrete:
    • A carefully configured MoE model with 0.85 billion active parameters matched a 6.1-billion-parameter dense model while using over seven times less training compute.
    • A looped 2.6-billion-parameter model matched standard transformers roughly three times its size.
    • A single change to how attention picks tokens let DeepSeek cut the price of processing 128,000 tokens from $2.30 to $0.30 per million.
A chatbot, a robot planner, and a fraud-screening pipeline need different architectures, and forcing one to act like another is how projects blow their budgets. Architecture determines:
    • What kinds of information the system can process (sequences, graphs, images, video, language, sensor streams, or physical environments).
    • How efficiently it learns, and how much labeled data it needs.
    • How much compute, memory, and energy it requires.
    • How easily it scales across machines.
    • How quickly it responds, which decides whether AI can sit inside real-time software.
    • Whether it can adapt to new tasks, keep learning after deployment, and generalize (see Out-of-Distribution (OOD) Generalization and Lifelong Learning).
    • Whether its outputs are reliable enough to act on without a human checking each one.
    • How easily it can be deployed, monitored, and maintained.

Recent Developments

The period from late 2024 through September 2026 reshaped almost every layer of AI architecture. This section covers what changed, what the evidence shows, and where researchers still disagree. It gives peer-reviewed work (Nature, NeurIPS, ICLR) extra weight and flags vendor-reported results as such.

Sparse experts became the default for open frontier models

Mixture-of-experts went from a clever trick to the standard way to build large open-weight language models, and the models got steadily sparser.

  • Mistral's Mixtral 8x7B (late 2023) had 46.7 billion total parameters with 13 billion active per token.
  • DeepSeek-V3 (December 2024) grew to 671 billion total parameters with only 37 billion active, roughly 1/18 of the model. It routes each token to 8 of 256 routed experts, plus always-on shared experts.
  • OpenAI's open-weight gpt-oss-20b uses 32 experts with top-4 routing, activating 3.6 billion of its 21 billion parameters.

Moonshot AI's Kimi K2 (July 2025) crossed the trillion-parameter line in the open. It has 1.04 trillion total parameters and about 32 billion active per token, routing each token to 8 of 384 experts. Its design is a deliberate descendant of DeepSeek-V3: fine-grained experts plus multi-head latent attention to keep the KV cache small. K2's bigger story, covered below, was how Moonshot kept a model that size stable during training.

DeepSeek V4, previewed in April 2026 and generally available by August 2026, pushed sparsity further:

  • V4-Pro has 1.6 trillion total parameters with 49 billion active per token (about 3 percent).
  • V4-Flash has 284 billion total with 13 billion active.
  • Both ship open weights under the MIT license with a one-million-token context window, and both pair sparse experts with a compressed sparse attention design that descends from DeepSeek Sparse Attention.

NVIDIA's Nemotron 3 Super (April 2026) shows the next step: combining MoE with a hybrid sequence architecture. It's a 120-billion-parameter model with 12 billion active, built from Mamba and attention layers. It's also the first in its family to use LatentMoE, which NVIDIA reports gives better accuracy per parameter and per FLOP than a regular MoE.

The field also got its first unified scaling law for MoE design. Ant Group's Ling team trained more than 300 models, up to 28 billion parameters, and introduced Efficiency Leverage: how much compute a dense model would need to match a given MoE. Their findings:

  • Leverage is driven mainly by the expert activation ratio and the total compute budget, both following predictable power laws.
  • Expert granularity acts as a non-linear modulator with a clear sweet spot.
  • Choices that engineers had debated for years, such as whether to use shared experts or how to arrange MoE and dense layers, turned out to have only a secondary effect.

The team validated its law with a pilot model of 0.85 billion active parameters. On the same one trillion tokens, it matched a 6.1-billion-parameter dense model while using over seven times less compute. The work was accepted at ICLR 2026.

Sparse attention made long contexts affordable

MoE made the feed-forward layers sparse. In late 2025, DeepSeek did the same for attention. DeepSeek Sparse Attention (DSA) debuted in DeepSeek-V3.2-Exp in September 2025 and carried into the full V3.2 release that December. It works in two stages:

  1. A tiny lightning indexer, which runs in FP8 with only a handful of heads, cheaply scores every earlier token for relevance to the current one.
  2. Full attention runs only over the top 2,048 scoring tokens.

DeepSeek trained the indexer on 2.1 billion tokens to predict where the model's original dense attention would focus.

The results matter because they're about money, not just benchmarks. DeepSeek reported that V3.2-Exp performed on par with its dense predecessor, V3.1-Terminus. Long-context processing costs fell by roughly six to seven times: from about $0.60 to $0.10 per million tokens at 32,000 tokens of context, and from $2.30 to $0.30 at 128,000. One analysis argued that the real point of DSA is to make it affordable for reasoning models to generate enormous numbers of thinking tokens.

DSA also sharpened an architectural fork in the road. Qwen3-Next replaced most attention layers with Gated DeltaNet layers, while DeepSeek kept full attention and made it sparse. Both approaches work, which suggests the transformer still has efficiency left to give.

Attention got a gate

One of 2025's most practical architecture results was also one of the simplest. Alibaba's Qwen team compared more than 30 variants of 15-billion-parameter MoE models and 1.7-billion-parameter dense models trained on 3.5 trillion tokens. Adding a head-specific sigmoid gate right after scaled dot-product attention consistently:

  • Improved performance.
  • Stabilized training.
  • Allowed larger learning rates.
  • Improved scaling.

The gate fixes a long-standing quirk called the attention sink. Softmax attention always has to put its probability mass somewhere, so a head with nothing useful to look at dumps that mass on the first token. The gate gives each head a way to say "nothing here is worth using." In the paper's analysis, gating cut the share of attention going to the first token from about 46.7 percent to about 4.8 percent. It also suppressed the massive activation outliers linked to loss spikes in low-precision training.

The paper won a NeurIPS 2025 Best Paper Award, one of four chosen from 5,290 accepted papers, and the design already ships in Qwen3-Next alongside Gated DeltaNet layers.

Optimizers joined the architecture conversation

For a decade, nearly every large model was trained with the AdamW optimizer, and optimizer choice looked like a detail. Kimi K2 changed that. Moonshot trained it on 15.5 trillion tokens with MuonClip, which combines three pieces:

  • Muon: an optimizer that orthogonalizes each gradient update and is known for its token efficiency.
  • RMS matching and weight decay: standard adjustments that keep update sizes consistent.
  • QK-Clip: a new technique that rescales a head's query and key weights whenever its largest attention logit crosses a threshold.

Exploding attention logits are a chief cause of the loss spikes that derail giant training runs. With QK-Clip, K2's entire pretraining finished without a single loss spike. One analysis describes K2 as the first open-weight, trillion-parameter MoE trained end to end with an optimizer other than AdamW, complete with a published recipe for keeping it stable.

The lesson connects to gated attention. Both results show that stability at scale is now designed into the model, whether through the architecture (a gate) or the optimizer (a clip), rather than patched in after a run fails.

Hybrid attention moved from experiment to production

The question "do we still need full attention in every layer?" now has a practical answer: no, but you probably need some of it. Several major 2025 and 2026 releases replaced most attention layers with cheaper recurrent layers that keep a fixed-size state.

  • Qwen3-Next 80B-A3B has 48 layers arranged as 12 repetitions of a 3:1 pattern, so 36 layers use Gated DeltaNet (a recurrent linear-attention method) and 12 keep (gated) softmax attention. Alibaba carried the same 3:1 schedule into its main Qwen3.5 line.
  • Kimi Linear from Moonshot AI uses 20 Kimi Delta Attention layers and 7 gated Multi-Head Latent Attention layers, roughly the same ratio.
  • NVIDIA Nemotron 3 Nano (30B-A3B) is a 52-layer model that interleaves Mamba-2 blocks with sparse MoE feed-forward layers and uses self-attention in only a small subset of layers.

The strongest controlled evidence comes from the Allen Institute for AI. In its OLMo Hybrid scaling experiments, the models ranked in this order:

  1. A 3:1 Gated DeltaNet hybrid.
  2. Pure Gated DeltaNet.
  3. A standard transformer.
  4. A Mamba-2 hybrid.
  5. Pure Mamba-2.

That result made Gated DeltaNet look like the clear winner among recurrent layers. Then the Mamba line fought back. Mamba-3, from Aakash Lahoti, Kevin Li, Berlin Chen, Caitlin Wang, Aviv Bick, Zico Kolter, Tri Dao, and Albert Gu, was an oral paper at ICLR 2026. It makes three changes:

  • An "exponential-trapezoidal" discretization that gives the recurrence more expressive dynamics.
  • A complex-valued state update that fixes linear models' well-known weakness at tracking state.
  • A multi-input, multi-output (MIMO) formulation that improves quality without increasing decoding latency.

At the 1.5-billion-parameter scale, Mamba-3 beat the next-best model, Gated DeltaNet, by 0.6 points of average downstream accuracy, and its MIMO variant added another 1.2 points. It also matched Mamba-2's perplexity with half the state size.

The fair reading is that the recurrent-layer race is still open, with results that shift with scale and setup. The bigger objection hasn't gone away either. Sebastian Raschka has pointed out that one of the transformer's proven strengths is how it performs at very large scale, and hybrids have yet to fully demonstrate that.

Reasoning moved into the training pipeline

The defining 2025 shift in language models wasn't a new layer type. It was a new training stage. DeepSeek-R1 showed that reasoning behaviors (self-reflection, verification, and switching strategies mid-problem) can be incentivized through pure reinforcement learning, without human-labeled reasoning traces. R1 became the most-downloaded model on Hugging Face for complex problem-solving after its January 2025 release. In September 2025 it became the first major LLM published in Nature after independent peer review, and Nature's editors urged other labs to follow that precedent.

The architectural lesson is subtle. R1 runs on the DeepSeek-V3 mixture-of-experts base, so "reasoning model" describes a training recipe and an inference behavior layered on top of an architecture, not a new architecture in itself. What changes is how compute is spent. Reasoning models generate long internal chains of thought before answering, which moves cost from training time to inference time. That makes KV-cache efficiency, sparse attention, and hybrid layers far more valuable.

TypeSafe frames the current training landscape as three philosophies:

  • RLHF, which optimizes for human preference.
  • Reinforcement learning with verifiable rewards (RLVR), which optimizes for outputs a program can check.
  • Its own RLCD, which optimizes for calibrated decisions.

Diffusion language models challenged left-to-right decoding

Autoregressive models write like a typewriter, one token at a time. Diffusion language models write like an editor: they start with a fully masked or noisy draft and refine every position in parallel over a fixed number of denoising steps. Inception Labs' Mercury models are transformers trained to predict many tokens at once, which keeps GPU arithmetic units saturated instead of waiting on memory. Because wall-clock latency scales with the number of denoising steps rather than output length, a diffusion model can produce a long answer in roughly the time an autoregressive model spends on a short one. Google DeepMind's Gemini Diffusion and the open LLaDA family are the other reference points.

The scale barrier fell in December 2025. Ant Group's LLaDA2.0 skipped training a diffusion model from scratch and instead converted a pretrained autoregressive model into a diffusion model. It used a three-phase schedule:

  1. Block sizes grow during warm-up.
  2. Full-sequence diffusion runs during the stable phase.
  3. Compact blocks return during decay.

The result was two open-source MoE diffusion models, LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B total, about 6.1B active). The flash model was continually trained on roughly 20 trillion tokens, and its model card reports an average benchmark score nearly identical to Qwen3-30B-A3B-Instruct.

The debate is about quality on hard reasoning. A NeurIPS 2025 theory paper argues that masked diffusion can hit near-optimal perplexity in a constant number of steps. Tasks that need the entire sequence to be correct, such as multi-step reasoning, may need a number of steps that grows with sequence length, which erodes the speed advantage. In practice, diffusion models look strongest today for latency-sensitive code and text generation, while autoregressive reasoning models still dominate the hardest reasoning benchmarks.

Memory and continual learning became an architecture problem

Standard transformers split memory crudely. The context window is short-term memory, and the weights frozen after pretraining are long-term memory. Nothing in between lets a model keep learning from experience without retraining. Google Research's Nested Learning (NeurIPS 2025) reframes a model as a set of nested optimization problems running at different speeds. It argues that architecture and optimizer are two views of the same thing, which the MuonClip story above illustrates from the practical side.

Its proof of concept, Hope, builds on Google's earlier Titans architecture, whose long-term memory modules prioritize what to remember based on how surprising it is. Hope adds two things:

  • A self-modifying recurrent design.
  • A "continuum memory system": a spectrum of memory modules that update at different frequencies, from fast to slow.

Google reports lower perplexity and higher accuracy than modern recurrent models and standard transformers on language-modeling and common-sense benchmarks. It also reports better recall on long-context needle-in-a-haystack tests. Independent readers describe the gains as modest but encouraging, and note that the authors don't claim to have solved continual learning. See Lifelong Learning and Memory Networks.

JEPA left the lab and became a company

The biggest organizational shift in architecture research was Yann LeCun's departure from Meta in November 2025 to pursue world models full time. His new company, Advanced Machine Intelligence (AMI Labs), announced a $1.03 billion seed round on March 10, 2026, at a $3.5 billion pre-money valuation. It was Europe's largest seed round on record, co-led by Cathay Innovation, Greycroft, Hiro Capital, HV Capital, and Bezos Expeditions. AMI is headquartered in Paris and is building JEPA-based world models for industrial, robotic, and healthcare applications.

Its CEO, Alexandre LeBrun, joked that "world models" would become the next buzzword every startup claims, and its first partner is the digital-health company Nabla. The leadership team includes Saining Xie as chief science officer, Pascale Fung as chief research and innovation officer, and Mike Rabbat as VP of world models.

Meta's FAIR team kept shipping JEPA research in parallel:

  • V-JEPA 2 (June 2025) is a 1.2-billion-parameter video model trained in two stages: action-free pretraining on video, then action-conditioned training.
    • It scored 77.3 percent on Something-Something v2, a motion-understanding benchmark.
    • It reached 39.7 percent on anticipating the next action one second ahead in Epic-Kitchens-100, a 44 percent improvement over earlier systems.
    • After training on just 62 hours of robot data from the Droid dataset, it could reach, grasp, and pick-and-place objects in new environments when given goal images. Success rates on pick-and-place of new objects ranged from 65 to 80 percent.
  • V-JEPA 2.1 (March 2026) fixes a known weakness: earlier JEPA models captured the global gist of a scene but blurred the fine-grained spatial detail robots need. It applies a dense predictive loss to all tokens instead of only the masked ones.
    • It set state-of-the-art scores for short-term object-interaction anticipation on Ego4D (7.71 mAP) and for action anticipation on EPIC-KITCHENS (40.8 Recall@5).
    • It improved real-robot grasping success by 20 points over V-JEPA 2-AC.
    • Meta released four encoders, ranging from an 80-million-parameter ViT-B to a 2-billion-parameter ViT-G.

On the theory side, LeJEPA (Randall Balestriero and Yann LeCun, November 2025) argues that an isotropic Gaussian is the optimal distribution for embeddings. It introduces SIGReg, a regularizer that pushes embeddings toward that shape. The practical payoff is simplicity. Earlier JEPAs needed a stack of heuristics to prevent "collapse," where every input maps to the same useless embedding: an exponential-moving-average teacher, stop-gradients, and capacity-limited predictors.

  • LeVJEPA, the first video encoder trained with LeJEPA's objective, drops all of that and reduces training to an encoder, a projector, and a single hyperparameter.
  • LeWorldModel applies the same recipe to an end-to-end world model trained directly from pixels.

Measuring physical understanding

A world model is only as good as the tests used to probe it, so Meta released three benchmarks alongside V-JEPA 2:

  • IntPhys 2 checks whether a model can tell physically plausible events from impossible ones, using the violation-of-expectation method developmental psychologists use with infants.
  • MVPBench uses minimally different video pairs so models can't pass by exploiting surface shortcuts.
  • CausalVQA asks counterfactual, hypothetical, anticipation, and planning questions about real-world video.

Humans score 85 to 95 percent across all three, and there's a notable gap between that and every tested model, including V-JEPA 2. The CausalVQA authors found that frontier multimodal models fall substantially below human performance, especially on anticipation and hypothetical questions. These benchmarks give the world-model debate something it lacked before: a shared, public yardstick for intuitive physics that doesn't reward language fluency.

The world-model debate: predict latents or generate worlds?

JEPA's central bet is that you shouldn't waste capacity predicting every pixel. The competing bet is that generating the world in full detail is itself the best training signal and the most useful product. Two systems anchor that camp:

  • Genie 3 from Google DeepMind (August 2025) generates interactive 3D environments from text in real time, at 720p and about 24 frames per second. It keeps them consistent for minutes and maintains object permanence without a separate memory module.
  • Marble from Fei-Fei Li's World Labs went generally available in November 2025. It turns text, images, video, or a rough 3D layout into a persistent, explorable 3D world. It exports both Gaussian splats for high-fidelity visuals and collision meshes a physics engine can use, so one model serves both viewing and robot simulation.

World Labs was founded by Li with Justin Johnson, Christoph Lassner, and NeRF co-author Ben Mildenhall. It raised $1 billion in February 2026 from investors including AMD, Autodesk (which committed $200 million), NVIDIA, Fidelity, Emerson Collective, and Sea, bringing its total to about $1.23 billion. In June 2026, the company argued in an essay that "world model" now covers three fundamentally different kinds of system.

Then, on September 28, 2026, AMD agreed to acquire World Labs for $8.2 billion in stock, with Li set to become AMD's chief scientist and an executive vice president. NVIDIA already offers its own open-weight world models, called Cosmos. The deal signals that chipmakers now see world models as strategic, not experimental.

Researchers are also stress-testing JEPA's own assumptions:

  • Is the Gaussian prior necessary? A 2026 study from Quantexa argues that anti-collapse pressure can come from the transition data itself. On one manipulation task with a movable cube, a SIGReg-based world model fell to a 58.0 percent three-seed average under the authors' planning protocol. Their contrastive inverse-dynamics alternative held at 80.0 percent.
  • Does it work for language? Another 2026 paper argues that SIGReg guarantees non-collapse but not predictability. On text, an encoder can pass every geometric collapse test and still fail to become semantically faithful, so non-collapse is necessary but not sufficient.
  • Does it produce full world models yet? Even V-JEPA 2.1's strongest supporters note that building complete world models on top of its dense features remains ongoing research.

The fair summary: JEPA has strong evidence for efficient representation learning and early evidence for robot planning. Generative world models have strong commercial traction and a clear role as training grounds for robots. Both camps still trail humans on intuitive physics. Whether latent prediction or generative simulation becomes the foundation for general-purpose world models is still an open, very well-funded experiment. For the logic of cause and effect behind these models, see Causation vs. Correlation.

Recursion versus scale

A striking 2025 result came from tiny models. The Hierarchical Reasoning Model (HRM) used two small networks recursing at different frequencies, with only 27 million parameters trained on about 1,000 examples. It beat large language models on Sudoku, mazes, and ARC-AGI puzzles. Then the ARC Prize Foundation ran an independent analysis. It found that HRM's gains came mostly from deep supervision (repeatedly refining an answer across training steps), not from its brain-inspired hierarchy.

Samsung researcher Alexia Jolicoeur-Martineau took that lesson to its conclusion. Her Tiny Recursive Model (TRM) drops the hierarchy and uses a single two-layer network with 7 million parameters. It reached 45 percent on ARC-AGI-1 and 8 percent on ARC-AGI-2, higher than many far larger LLMs.

The caveats are real:

  • TRM's pretraining needed four H100 GPUs for over 48 hours, far beyond the ARC Prize 2025 competition's compute limit.
  • A follow-up technical note set out to measure how much of TRM's performance comes from architecture versus test-time compute and task-specific priors.
  • These models are task-specific: a network trained on Sudoku doesn't transfer to ARC without retraining.

The same idea then crossed into general-purpose language models. In October 2025, a team led by ByteDance Seed, with collaborators including Yoshua Bengio and Jason Eshraghian, released Ouro, a family of looped language models. Ouro builds reasoning into pretraining instead of bolting it on later:

  • A shared block of transformer layers runs up to four times in a row.
  • A learned exit gate decides how many passes each input needs.
  • The models were pretrained on 7.7 trillion tokens.

The 1.4-billion and 2.6-billion-parameter Ouro models matched standard transformers of 4 billion and 8 billion parameters across nearly all benchmarks. Controlled experiments showed the gain didn't come from storing more knowledge (both looped and standard models hold about 2 bits per parameter). It came from being better at combining and manipulating what they know. The authors also found that looped reasoning produced internal traces that matched final answers more faithfully than explicit chain of thought.

Taken together, these results reframe the question from "how big?" to "how many refinement steps?" See also Chain of Thought, which gets extra reasoning depth by generating text rather than by looping in latent space.

System One decision models

In September 2026, TypeSafe AI launched Jev, a transformer-based model that returns typed decisions with probabilities instead of generated text. Architecturally, it matters because it treats decision-making as its own model class, with its own training objective (calibration rather than human preference) and its own sampler (parallel rather than autoregressive).

The category then spread unusually fast. Within two weeks, open reimplementations such as Kev and Laya had appeared, OpenAI had announced a Decisions API built on GPT-6 Luna, and Cloudflare (Clef and Clef-flash), Perplexity (pplx-decider), and AWS (Strands Decider 2B) had shipped decision models of their own, most of them built on Alibaba's Qwen models. The architecture is summarized under System One Decision Models in Technical Architecture. The whole category, including how to use these models and how the offerings compare, is covered in the Decision Models part below.

Neuromorphic hardware matured, slowly

Neuromorphic computing hasn't displaced GPUs, but it has produced credible large-scale results. Intel's Hala Point, delivered to Sandia National Laboratories in 2024, packs 1,152 Loihi 2 chips into a six-rack-unit chassis with 1.15 billion artificial neurons and 128 billion synapses. Sandia researchers fit the entire FlyWire fruit-fly connectome onto just 12 Loihi 2 chips and ran it more than 100 times faster than numerical simulations on conventional hardware.

In January 2025, a 23-author review in Nature laid out a roadmap for the field, describing it as at a critical juncture. The lead author is Dhireesha Kudithipudi of the University of Texas at San Antonio, and the authors include SpiNNaker architect Steve Furber. They argued that scale is the key metric to track and that systems will need to grow considerably larger to handle complex real-world problems. Kudithipudi compared the moment to deep learning's AlexNet breakthrough. The open question is whether neuromorphic gains transfer from brain simulation and event-driven sensing to mainstream deep learning workloads.

What's still unsettled

  • Will hybrid sequence architectures hold their advantage at the very largest scales, or will full attention (made cheaper by sparse attention) win back ground?
  • Which recurrent layer wins: Gated DeltaNet, which led Ai2's scaling study, or Mamba-3, which beat it at 1.5 billion parameters?
  • Now that MoE efficiency has a scaling law, how far can activation ratios fall before routing and communication costs erase the gains?
  • Will optimizers like MuonClip replace AdamW for frontier training, and do they favor certain architectures?
  • Can diffusion language models match autoregressive models on long, multi-step reasoning without losing their speed advantage?
  • Will looped language models like Ouro keep their two to three times parameter efficiency at frontier scale?
  • Will learned memory modules like Titans and Hope reduce the need for external retrieval, or complement it?
  • Will latent-prediction world models (JEPA) or generative world models (Genie 3, Marble) close the gap to humans on intuitive-physics benchmarks first?
  • How much of the recursive-reasoning gain comes from architecture versus extra compute and task priors?
  • Will specialized decision models such as Jev, Clef, and pplx-decider become a standard layer in software, or will general LLMs absorb that role as they get cheaper? OpenAI's Decisions API, which constrains GPT-6 Luna instead of training a separate model, is an early test of the second path.

Technical Architecture

A Modern AI System

A simplified AI architecture looks like a series of layers: 

                    ┌───────────────────────┐
                    │        Users          │
                    │ People / Apps / Agents│
                    └───────────┬───────────┘
                                │
                                ▼
                    ┌───────────────────────┐
                    │   Interface Layer     │
                    │ Text / Voice / Vision │
                    │ API / Robot / Sensors │
                    └───────────┬───────────┘
                                │
                                ▼
                    ┌───────────────────────┐
                    │ Application / Agent   │
                    │ Orchestration Layer   │
                    │ Routing / Tools /     │
                    │ Policies / Workflows  │
                    └───────────┬───────────┘
                                │
               ┌────────────────┼────────────────┐
               │                │                │
               ▼                ▼                ▼
       ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
       │ Core Models  │ │    Memory    │ │    Tools     │
       │ Transformer  │ │ Context      │ │ Search       │
       │ MoE / JEPA   │ │ Retrieval    │ │ APIs         │
       │ GNN / Hybrid │ │ Databases    │ │ Applications │
       │ SSM/Decision │ │ Vector store │ │ Code runners │
       └──────┬───────┘ └──────────────┘ └──────────────┘
              │
              ▼
       ┌─────────────────────────┐
       │ Compute & Infrastructure│
       │ GPU / TPU / CPU / Edge  │
       │ Distributed Computing   │
       │ Containers / Serverless │
       └─────────────┬───────────┘
                     │
                     ▼
              Output / Action

Each layer can use a different architectural approach. The orchestration layer is where most of the recent action is: it decides which model handles a request, what context reaches it, and whether its output gets checked before anything happens. See Messaging & Routing.

Transformer Architectures

Transformers became the dominant architecture for language and, increasingly, for vision, audio, video, and multimodal AI.

Instead of processing information strictly one step at a time, transformers use attention to decide which parts of the input matter most to each other part. Every token can "look at" every other token in a single layer, which is why transformers train so efficiently in parallel on modern hardware.

Important transformer components include:

  • Token or input embeddings.
  • Positional information.
  • Self-attention.
  • Multi-head attention.
  • Feed-forward layers.
  • Residual connections.
  • Normalization.
  • Encoder, decoder, or encoder-decoder arrangements.

Transformer systems come in three main shapes:

Encoder-only
Optimized for understanding or representation tasks, such as classification, retrieval, and embeddings.
Decoder-only
Common in generative large language models, where the model predicts subsequent tokens.
Encoder-decoder
Useful when one sequence must be transformed into another, such as translation or structured generation.

The weak spot is cost. Full attention grows quadratically with sequence length, and during generation the key-value (KV) cache grows with every token of context. Modern implementations fight back in several ways:

  • Grouped-query attention, which shares key and value heads across query heads.
  • Multi-head latent attention (MLA), which compresses keys and values into a smaller latent space to shrink the KV cache.
  • Sparse attention, which attends only to the most relevant positions.
  • Optimized KV caches and alternative positional encodings.

When those tricks aren't enough, designers swap most attention layers for state-space or linear-attention layers (see below).

DeepSeek Sparse Attention is the best-documented example of learned sparse attention. It's layered on top of MLA and works like a fast filter in front of expensive attention: 

   All earlier tokens (for example, 128,000)
                    │
                    ▼
   Lightning indexer: a few FP8 heads
   give every token a cheap relevance score
                    │
                    ▼
   Keep only the top 2,048 tokens
                    │
                    ▼
   Full attention over the selected tokens

The indexer technically still looks at every token. Because it's tiny and runs in low precision, though, its cost is dwarfed by the savings from skipping full attention on the rest of the context.

Gated attention is the newest standard refinement. After computing attention, each head multiplies its output by a learned, input-dependent sigmoid gate between 0 and 1: 

   Standard:   output = SDPA(Q, K, V)
   Gated:      output = sigmoid(X · W_gate) ⊙ SDPA(Q, K, V)

   gate near 0 → "nothing useful here, pass nothing"
   gate near 1 → "pass the attention result through"

That small change gives heads a way to decline to contribute. As a result, they stop dumping unused attention on the first token (the "attention sink") and stop propagating the activation spikes that destabilize low-precision training. Qwen's systematic study found it improves stability, tolerates larger learning rates, and scales better. Sparse attention and gating aren't rivals: researchers have already begun combining the two, pairing sparse attention's efficiency with gating's stability.

Mixture-of-Experts Architecture

Mixture-of-Experts (MoE) increases model capacity without activating every parameter for every input.

Instead of sending every token through one large feed-forward network, an MoE layer contains many specialized experts. A learned router decides which experts should process each token. 

                      Input
                        │
                        ▼
                     Router
               ┌────────┼────────┐
               ▼        ▼        ▼
            Expert 1 Expert 2 Expert 3 ... Expert N
               │        │
               └────┬───┘
                    ▼
                  Output

Only a subset of experts is active for any one token. That produces a sparse architecture: the model can hold an enormous number of total parameters while spending compute on only a small fraction of them per step.

The dominant modern pattern splits experts into two kinds. Shared experts process every token and capture general knowledge. Routed experts are selected per token and specialize. DeepSeek-V3, for example, combines shared experts with 256 routed experts and activates 8 routed experts per token. Over the past two years, the trend has been toward more, smaller experts and lower activation ratios:

Model Total parameters Active per token Notes
Mixtral 8x7B (2023) 46.7B 13B Matched a much larger dense model while running faster
DeepSeek-V3 (2024) 671B 37B Shared plus routed experts, 8 of 256 routed experts active
Kimi K2 (2025) 1.04T 32B 8 of 384 experts per token; trained with the MuonClip optimizer
gpt-oss-20b (2025) 21B 3.6B 32 experts, top-4 routing
Qwen3-Next 80B-A3B (2025) 80B 3B Also a hybrid, gated-attention model
LLaDA2.0-flash (2025) 100B 6.1B A diffusion language model, not autoregressive
Nemotron 3 Super (2026) 120B 12B LatentMoE on a hybrid Mamba-attention backbone
DeepSeek V4-Flash (2026) 284B 13B Open weights, 1M-token context
DeepSeek V4-Pro (2026) 1.6T 49B About 3 percent of parameters active per token

What actually drives MoE efficiency? According to Ant Group's Efficiency Leverage scaling law, three settings matter most:

  • The fraction of experts active per token. Lower ratios give more leverage, following a power law.
  • The total compute budget. MoE's advantage grows as training compute grows.
  • Expert granularity. It has a clear optimal range rather than "smaller is always better."

Shared experts and the arrangement of MoE and dense layers turned out to be second-order effects with near-optimal defaults.

Advantages include:

  • Greater model capacity for the same per-token compute.
  • Computational efficiency relative to similarly sized dense models, with an advantage that grows with compute budget.
  • Specialization among experts.
  • Scalability across distributed hardware through expert parallelism.

Challenges include:

  • Routing efficiency and router collapse (a few experts getting all the traffic).
  • Load balancing among experts.
  • All-to-all communication between accelerators.
  • Training stability, especially at trillion-parameter scale (see below).
  • Memory requirements: every expert must be stored, even if few are used per token.

Training Stability and Optimizer Co-Design

The bigger a model gets, the more likely a training run is to blow up. Attention logits (the raw scores before softmax) can grow without bound, and a single spike can waste weeks of compute. Since 2025, fixes for this problem have moved into the architecture and the optimizer themselves.

Technique Where it acts What it prevents
Gated attention Architecture: a sigmoid gate after each attention head Attention sinks and massive activation outliers that cause loss spikes in low precision
QK-Clip (part of MuonClip) Optimizer: rescales query and key weights after each update when the largest logit crosses a threshold Exploding attention logits in very large runs
Muon Optimizer: orthogonalizes each gradient update Wasted training tokens (it's more token-efficient than AdamW)
Low-precision formats (FP8, NVFP4) Numerics High memory and bandwidth cost, but they demand the stability fixes above

Kimi K2 is the proof point. Its full 15.5-trillion-token run completed with zero loss spikes using MuonClip. The broader lesson echoes Google's Nested Learning argument: architecture and optimizer are two sides of one design problem, and modern model cards should be read with both in mind.

State-Space and Linear-Attention Hybrids

State-space models (SSMs) and linear-attention layers process sequences by updating a fixed-size internal state, much like a recurrent network, instead of keeping a growing cache of every past token. That makes their cost grow linearly with sequence length and keeps memory flat during generation. Linear attention gets there by reordering the attention math: because matrix multiplication is associative, you can combine keys and values first and then apply queries, which removes the quadratic term.

The catch is that a fixed-size state is lossy. Pure SSMs historically struggled with tasks that need precise recall of something far back in the context, like copying or exact lookup, and with tracking how a state changes over many steps. The practical fix is a hybrid: mostly cheap recurrent layers, with occasional full-attention layers to handle precise retrieval. 

   Layer pattern in a typical 3:1 hybrid

   ┌────────────┐
   │ Linear/SSM │  cheap, fixed-size state
   ├────────────┤
   │ Linear/SSM │
   ├────────────┤
   │ Linear/SSM │
   ├────────────┤
   │ Attention  │  precise recall, growing cache
   └────────────┘
        × N repetitions

The main families of recurrent layers are:

  • Mamba and Mamba-2 (selective state-space models), used in NVIDIA's Nemotron 3 family and IBM's Granite 4.
  • Mamba-3 (ICLR 2026), which adds a more expressive discretization, complex-valued states for better state tracking, and a multi-input, multi-output formulation. MIMO raises arithmetic intensity so the GPU does more useful work per memory access, without slowing down decoding. It works both on its own and inside hybrid stacks.
  • Gated DeltaNet and its variants, which update the state with a learned "delta rule" and decay gates. They're used in Qwen3-Next, Qwen3.5, Kimi Linear, and Ai2's OLMo Hybrid.

See Recent Developments above for the scaling evidence. The short version: hybrids are now production-grade. Gated DeltaNet hybrids have the strongest large-scale published results, and Mamba-3 currently leads at the 1.5-billion-parameter scale.

Diffusion Language Models

Diffusion language models borrow the idea behind image diffusion. Instead of predicting the next token, they start with a sequence of masked or noisy positions and run a fixed number of refinement passes. Each pass predicts all positions at once and commits the most confident ones. 

Autoregressive                      Diffusion
Step 1:  The                        Step 1:  [M] [M] [M] [M] [M]
Step 2:  The cat                    Step 2:  The [M] sat [M] [M]
Step 3:  The cat sat                Step 3:  The cat sat on  [M]
Step 4:  The cat sat on             Step 4:  The cat sat on  mats
Step 5:  The cat sat on mats
(one token per step)                (all positions refined each step)

Because every pass works on every position, the hardware does more useful arithmetic per byte of memory moved, and latency depends on the number of steps rather than output length. The tradeoffs:

  • Each step is more expensive than a single autoregressive step.
  • Tasks that need the whole sequence to be exactly right may need more steps, which narrows the speed gap.

A practical middle ground is block diffusion, which generates text in blocks left to right while refining each block in parallel. LLaDA2.0 relies on it to convert existing autoregressive models into diffusion models instead of training them from scratch. That means diffusion can inherit the knowledge already paid for in pretraining.

Reasoning Models and Test-Time Compute

A reasoning model is a base model (dense or MoE) that has been further trained to think before answering. It generates an internal chain of intermediate steps, checks its own work, and backtracks when needed. The typical pipeline looks like this: 

   Pretrained base model (dense or MoE)
                 │
                 ▼
   Optional supervised "cold start" data
                 │
                 ▼
   Large-scale reinforcement learning
   with automatically checkable rewards
   (math answers, passing unit tests)
                 │
                 ▼
   Reasoning model: spends extra tokens
   "thinking" before it answers

DeepSeek-R1 is the peer-reviewed reference point: pure reinforcement learning produced self-reflection, verification, and dynamic strategy changes without human-written reasoning examples.

Reasoning shifts the cost profile of an architecture. A model that writes thousands of hidden tokens per answer is dominated by decoding cost, so memory bandwidth, KV-cache size, and attention efficiency suddenly matter more than raw parameter count. That's a big reason why sparse attention, gated attention, and hybrid layers spread so quickly in 2025 and 2026.

There are now four distinct ways to buy extra "thinking" at inference time:

  • Text-based reasoning: generate a longer chain of thought (see Chain of Thought and Tree of Thoughts).
  • Looped language models: run the same block of layers several times before emitting each token, with a learned gate deciding when to stop (Ouro).
  • Latent recursion on small task models: loop a tiny network many times on a single puzzle (HRM and TRM; see Hierarchical and Recursive Architectures below).
  • Parallel refinement: run more denoising passes in a diffusion model.

System One Decision Models

Reasoning models buy accuracy by thinking longer. Decision models make the opposite trade: they give up text generation entirely and answer narrow, typed questions in tens to hundreds of milliseconds. TypeSafe AI introduced the class with Jev in September 2026 and calls these "System One" models, the fast, intuitive counterpart to slow System 2 reasoning. 

   Generative LLM                          Decision model

   Prompt + context                        State + typed questions
          │                                         │
          ▼                                         ▼
   Generate tokens one at a time           One parallel pass over all questions
          │                                         │
          ▼                                         ▼
   Text to parse, validate, retry          Typed answers + probabilities
   (can drift off-schema)                  (always inside the declared answers)

Three architectural choices define the class:

  • What it predicts: a probability distribution over answers the caller declares at request time, through three question types (Noul for yes/no, Choice for one-of-N, and Score for an ordered rubric), instead of a token sequence.
  • How it computes: a single prefill-style pass with no autoregressive decoding. The state is encoded once and each question runs as an isolated branch, so adding questions barely adds latency.
  • How it's trained: for calibration (TypeSafe's RLCD, or cross-entropy and Brier-score losses in open models) rather than human preference.

Implementations reach the same interface by different routes: a purpose-built model (Jev), an encoder with a decision head (Laya), an open LLM with LoRA adapters and a pointer head (Kev, Strands Decider, Clef), logit readouts from an unchanged LLM (AnyJev, SemIf), a diffusion canvas (DiffusionGemma), or a constrained frontier model (OpenAI's Decisions API). See Computational routes.

Decision models complement generative models rather than replace them. They sit around a frontier LLM as routers, verifiers, and guardrails, which makes them a building block for cascades and hybrid systems. The Decision Models part below covers how they work, where they fit in a system, their limits, how to evaluate them, and every major offering.

Memory and Continual-Learning Architectures

Most deployed models have only two kinds of memory: what fits in the context window, and what was frozen into the weights during training. Everything else, such as documents, user history, and new facts, gets bolted on through retrieval. Researchers are now building memory into the architecture itself. 

   Fast ◄──────────────────────────────────────────────► Slow

   Attention /     Learned long-term     Continuum memory    Pretrained
   KV cache        memory module         (many update        weights
   (this prompt)   (Titans: keeps        speeds, Hope)       (frozen after
                   surprising facts)                          training)
  • Context and KV cache: precise but temporary, and expensive at long lengths. See In-Context Learning (ICL).
  • External retrieval: vector stores and knowledge graphs queried at run time. It's cheap and updatable, but the model doesn't actually learn anything.
  • Learned memory modules: Google's Titans adds long-term memory modules that decide what to keep based on how surprising new information is.
  • Continuum memory: Hope, from Google's Nested Learning work, uses a spectrum of memory modules updating at different frequencies and can modify its own memory through a self-referential process.

The goal is continual learning without catastrophic forgetting: a system that keeps improving from experience after deployment. Early results are promising but modest, so treat these as research architectures rather than production defaults. Earlier ideas in this lineage include Memory Networks and Hierarchical Temporal Memory (HTM). See also Lifelong Learning.

Joint Embedding Predictive Architecture (JEPA)

Joint Embedding Predictive Architecture (JEPA) is a family of self-supervised architectures proposed by Yann LeCun in 2022. It has been developed by researchers at Meta FAIR and, since 2026, at AMI Labs.

JEPA takes a different route from systems that learn by reconstructing raw inputs or predicting the next token or pixel. The central idea:

Predict the representation of missing or future information, not the information itself.

In a simplified JEPA: 

              Observed Input
                    │
                    ▼
             Context Encoder
                    │
                    ▼
            Context Embedding
                    │
                    ▼
                Predictor
                    │
                    ▼
          Predicted Representation
                    │
                    │ compare
                    ▼
            Target Representation
                    ▲
                    │
             Target Encoder
                    ▲
                    │
              Target Input

The encoders convert observations into abstract numerical representations called embeddings. The predictor tries to infer the embedding of a hidden, missing, or future part of the input, and the system learns by pulling its prediction close to the target embedding.

Why bother? Because many details of the physical world are impossible or pointless to predict exactly. Imagine watching a person walk behind a parked truck. A pixel-generating system would try to predict exact colors, shadows, textures, clothing movement, background details, and every possible future pixel. A JEPA-style system can instead learn the useful prediction at a higher level:

The person keeps moving and will probably come out the other side of the truck.

The architecture concentrates learning on predictable structure and ignores noise it could never predict anyway.

The hard engineering problem is collapse. If the encoder maps every input to the same embedding, prediction becomes trivially perfect and completely useless. Every JEPA needs some anti-collapse mechanism, and the history of the family is largely a history of better solutions to that problem.

I-JEPA

I-JEPA (2023) applies JEPA to images. Parts of an image are hidden from the context encoder, and the model predicts the embeddings of the hidden patches from the visible ones. It never tries to redraw the missing pixels.

V-JEPA

V-JEPA (2024) extends the approach to video. Instead of predicting exact future frames, it learns representations of objects, motion, interactions, and events by predicting masked regions of a video's representation. That makes it well suited to learning how the physical world changes over time.

V-JEPA 2

V-JEPA 2 (June 2025) turns the architecture into a world model that can:

  • Understand observed environments.
  • Predict how situations may change.
  • Anticipate actions.
  • Plan toward a goal.
  • Support robotic control.

It's a 1.2-billion-parameter model trained in two stages. Self-supervised pretraining on large amounts of video comes first, followed by action-conditioned training that teaches it how actions change the environment. Aligned with a language model, it also set state-of-the-art results on video question-answering benchmarks such as Perception Test and TempCompass.

For robot planning, the system compares three things:

  1. The representation of the robot's current state.
  2. The representation of a goal image.
  3. Predicted representations that would result from candidate actions.

The robot picks the action expected to move the world closest to the goal, executes it, observes the result, and re-plans at every step. This is a standard model-predictive-control loop. For longer tasks, it chases a sequence of visual subgoals. Trained on only 62 hours of robot data, this was enough for reaching, grasping, and pick-and-place in unfamiliar environments, with 65 to 80 percent success on pick-and-place of new objects.

V-JEPA 2.1

V-JEPA 2.1 (March 2026) adds the fine-grained spatial detail that manipulation needs. It applies its predictive loss densely to all tokens rather than only masked ones, produces unified image and video representations, and improves real-robot grasping success by 20 points over V-JEPA 2-AC.

LeJEPA and SIGReg

LeJEPA replaces the old anti-collapse heuristics with a single statistical regularizer, SIGReg, which pushes embeddings toward an isotropic Gaussian distribution that the authors argue is optimal. Follow-up work applies it to video (LeVJEPA) and to end-to-end world models from pixels (LeWorldModel). Critics argue that the Gaussian constraint isn't always the best source of anti-collapse pressure.

Benchmarks for physical reasoning

Meta's three physical-reasoning benchmarks (IntPhys 2, MVPBench, and CausalVQA) give world models a common test. Humans score 85 to 95 percent, and current models, including V-JEPA 2, fall well short. The hardest categories are anticipating what happens next and reasoning about hypotheticals. A public leaderboard tracks progress against human scores, which makes these benchmarks the most direct way to compare JEPA-style and generative world models on the same footing.

Generative versus JEPA world models

Generative World Model (Genie 3, Marble)
Observed State or Prompt
      │
      ▼
Generate Future Frames / 3D Scene
      │
      ▼
Agent or Robot Acts Inside the Simulation


JEPA World Model (V-JEPA 2, LeWorldModel)
Observed State
      │
      ▼
Encode Abstract State
      │
      ▼
Predict Future Abstract State
      │
      ▼
Compare with Goal
      │
      ▼
Choose Action

The two approaches trade off differently:

  • Generative models produce something humans and physics engines can inspect directly. Marble, for example, outputs collision meshes a simulator can use. The cost is that they spend capacity on every visible detail.
  • JEPA models are cheaper to plan with because they never render anything, but their predictions are harder to inspect.

Both are particularly relevant to robotics, autonomous systems, video understanding, and any system that has to reason about physical cause and effect.

Official resources:

Graph Architectures

Graph Convolutional Network (GCN), Graph Neural Networks (Graph Nets), Geometric Deep Learning are designed for information represented as connected entities rather than simple sequences or grids.

Examples include:

  • Social networks.
  • Molecules.
  • Transportation networks.
  • Knowledge graphs.
  • Recommendation systems.
  • Biological networks.
  • Communication networks.

A graph architecture represents entities as nodes and relationships as edges. Information then propagates between neighboring nodes through repeated rounds of "message passing."

That lets the system learn not only from individual objects but from the relationships connecting them. In agent systems, knowledge graphs increasingly serve as structured memory that a language model can query instead of relying only on text retrieval.

Convolutional and Capsule Architectures

Convolutional neural networks remain important in image processing, signal processing, edge systems, and specialized computer vision. Convolutions bake in a useful assumption, that nearby pixels or samples matter most, which makes them efficient and data-friendly.

Capsule Networks (CapNets) were proposed as an alternative for representing hierarchical spatial relationships between features.

Transformers have taken over many tasks once dominated by convolution, but convolution remains highly useful where efficiency, spatial locality, or specialized hardware matter. Many modern vision and speech systems are hybrids that use convolutional layers early and attention layers later. See also Convolution vs. Cross-Correlation (Autocorrelation).

Ensemble Architectures

An ensemble combines multiple models instead of depending on a single one.

Common strategies include:

  • Averaging model outputs.
  • Voting.
  • Weighted voting.
  • Bagging.
  • Boosting.
  • Stacking.
  • Model routing.
  • Specialized-model selection.
  • Cascades, where a cheap model answers first and an expensive model only handles flagged cases.

The models may share an architecture or use entirely different ones. 

                    Input
                      │
        ┌─────────────┼─────────────┐
        ▼             ▼             ▼
      Model A       Model B       Model C
        │             │             │
        └─────────────┼─────────────┘
                      ▼
               Aggregation Layer
                      │
                      ▼
                 Final Output

Ensembles improve reliability and accuracy when the individual models' errors aren't perfectly correlated. The tradeoff is extra compute, memory, orchestration, and latency.

Cascades are the modern, cost-aware version of the idea. Instead of running every model on every input, a fast verifier decides when a second opinion is worth paying for. TypeSafe's published cascade is a good worked example: a small model extracts data, Jev checks each field, and only flagged records go to an expensive reasoning model. Independent studies show the catch: a cascade only beats its fallback when the cheap model is more accurate on the cases it accepts, and escalation can't fix errors the fallback model shares (see Cascades and escalation).

Hybrid Architectures

Hybrid architectures combine different computational approaches within one system.

Examples include:

  • Transformer + convolution.
  • Transformer + state-space model (Qwen3.5, Kimi Linear, Nemotron 3).
  • Transformer + Mixture-of-Experts (MoE).
  • Dense attention + learned sparse selection (DeepSeek Sparse Attention).
  • Diffusion + mixture-of-experts (LLaDA2.0).
  • Neural network + symbolic reasoning.
  • Neural network + search, as in Google DeepMind AlphaGo Zero, which paired a deep network with tree search.
  • Language model + graph processing.
  • Generative model + predictive world model.
  • Generative world model + physics engine (Marble's collision meshes).
  • Frontier LLM + fast decision model (such as Jev, Clef, or pplx-decider; see Decision Models).
  • Cloud model + edge model.
  • Neural model + external search and tools.

Hybrids are now the norm because different components are good at different jobs. The goal isn't architectural purity. It's an effective division of work.

Hierarchical and Recursive Architectures

A hierarchical architecture divides processing into levels. Lower levels handle detailed or immediate information, while higher levels represent more abstract goals, concepts, or plans. 

             Strategic Goal
                   │
                   ▼
              Task Planning
                   │
                   ▼
             Action Planning
                   │
                   ▼
              Motor Action
                   │
                   ▼
                Sensors

Hierarchical designs are especially relevant to robotics, planning, autonomous systems, and long-horizon reasoning. See also Hierarchical Temporal Memory (HTM).

Recursive architectures get depth by reusing the same small network many times instead of stacking more layers. The 2025 Hierarchical Reasoning Model and Tiny Recursive Model showed that a few million parameters, looped many times with the answer refined at each step, can beat much larger LLMs on certain puzzles. The key lesson from the follow-up analysis is humbling for hierarchy fans: the repeated refinement mattered more than the brain-inspired hierarchy itself. 

   Recursive refinement (TRM-style)

   Input x ──┐
             ▼
   ┌───────────────────┐
   │  Tiny network     │◄──┐
   │  update latent z  │   │  repeat N times
   └─────────┬─────────┘   │
             ▼             │
   ┌───────────────────┐   │
   │ update answer y   │───┘
   └─────────┬─────────┘
             ▼
        Final answer

Looped language models bring the same idea to general-purpose LLMs. Instead of a separate puzzle network, a standard decoder-only transformer applies one shared stack of layers several times per token, and a learned exit gate stops early when extra passes won't help. 

   Looped language model (Ouro-style)

   Token embeddings
          │
          ▼
   ┌─────────────────────┐
   │ Shared layer stack  │◄──┐
   └──────────┬──────────┘   │  up to 4 passes
              ▼              │
      Exit gate: done? ──no──┘
              │ yes
              ▼
        Next-token output

ByteDance's Ouro models showed that this buys depth without adding parameters. The gain comes from better knowledge manipulation rather than more knowledge storage, which is exactly what reasoning needs.

Neuromorphic and Bio-Inspired Architectures

Neuromorphic Computing and Bio-inspired Computing explore architectures influenced by biological nervous systems.

These approaches may use:

  • Spiking neural networks.
  • Event-driven computation.
  • Distributed processing.
  • Local memory and computation.
  • Low-power hardware.
  • Adaptive connectivity.

The goal isn't to reproduce the brain literally. Biology offers design ideas for machines that process information efficiently, adapt continuously, and run under strict energy limits.

The flagship research system today is Intel's Hala Point, with 1.15 billion artificial neurons across 1,152 Loihi 2 chips. Sandia National Laboratories has used it for tasks such as running the full fruit-fly connectome far faster than conventional simulation. The field's 2025 Nature roadmap argues that neuromorphic systems must scale much further, and that they need a shared ecosystem of hardware, software, and benchmarks, to matter for real-world AI workloads.

See also Neuroscience, Symbiotic Intelligence, and Evolutionary/Genetic.

Decentralized and Distributed Architectures

Decentralized: Federated & Distributed architectures divide AI workloads across multiple systems, devices, data centers, or organizations.

Distribution is now fundamental to large-scale AI because training and serving modern models requires many processing units working together.

Common approaches include:

  • Data parallelism.
  • Model parallelism.
  • Pipeline parallelism.
  • Tensor parallelism.
  • Expert parallelism (placing different MoE experts on different accelerators).
  • Federated learning.
  • Distributed inference.
  • Edge and cloud cooperation.

Golem is one example of a distributed-computing network.

See also Service Capabilities and Processing Units - CPU, GPU, APU, TPU, VPU, FPGA, QPU.

Technologies Implemented

Modern AI systems frequently combine several architectural technologies.

Architecture / Technology Primary Function Typical Use
Transformer Uses attention to model relationships across input elements Language, multimodal AI, vision, audio, video, code
Gated Attention Lets each attention head scale down or switch off its output Stable large-scale training, long-context LLMs
Learned Sparse Attention Scores all earlier tokens cheaply, then attends only to the most relevant ones Long-context and long-reasoning LLMs at lower cost
Encoder Architecture Converts input into useful representations Classification, retrieval, embeddings, perception
Decoder Architecture Generates sequences from previous context Large language models, text generation, code generation
Encoder-Decoder Maps one representation or sequence into another Translation, summarization, multimodal transformation
Mixture-of-Experts (MoE) Routes inputs to selected specialized subnetworks Large-scale sparse models
Stability-Aware Optimizers Train giant models without loss spikes (for example, MuonClip) Trillion-parameter pretraining
State-Space and Linear-Attention Hybrids Replace most attention layers with a fixed-size recurrent state Long-context LLMs, high-throughput inference
Diffusion Language Models Generate by refining all positions in parallel over several steps Low-latency text and code generation
Reasoning Models Use reinforcement learning to think step by step before answering Math, coding, science, agentic planning
Looped Language Models Reuse one layer stack several times per token with a learned exit Parameter-efficient reasoning, small deployable models
Learned Memory Modules Store and update information at multiple time scales inside the model Long-context recall, continual learning research
Joint Embedding Predictive Architecture Predicts hidden or future information in representation space Self-supervised learning, vision, video, world models, robotics
Generative World Models Generate explorable, persistent environments from prompts or images Simulation, robot training, gaming, visual effects
Recursive Reasoning Models Reuse a small network many times to refine an answer Puzzles, constraint satisfaction, ARC-style tasks
System One Decision Models Return typed decisions with calibrated probabilities instead of text Routing, classification, verification, guardrails, agent action gating
Convolutional Networks Learns spatially local patterns Vision, signals, edge AI
Graph Neural Networks Learns from entities and relationships Molecules, networks, recommendations, knowledge graphs
Capsule Networks Represents hierarchical spatial relationships Experimental and specialized computer vision
Ensembles and Cascades Combine or sequence predictions from multiple models Reliability, classification, forecasting, cost control
Hybrid Architectures Combines different model types Multimodal systems, agents, reasoning, specialized AI
Hierarchical Architectures Organizes computation at different levels of abstraction Planning, robotics, autonomous systems
Neuromorphic Architecture Uses event-driven and brain-inspired computation Low-power AI, robotics, sensor processing, brain simulation
Distributed Architecture Divides computation or learning across machines Large-model training, inference, federated AI
Memory Maintains information beyond the immediate model calculation Agents, personalization, retrieval, persistent systems
Accelerated Compute Executes training and inference workloads GPUs, TPUs, specialized AI accelerators
Containers & Microservices Packages and orchestrates system components Production deployment, scaling, model services
Serverless Runs application functions without dedicated persistent servers APIs, event processing, AI workflows

Decision Models

This part is the in-depth guide to decision models. For a short architectural summary, see System One Decision Models in Technical Architecture.

A decision model is an AI model that reads a piece of context, called the state, together with one or more typed questions about it, and returns a typed answer drawn from an answer space the caller declares in advance, along with a probability for every allowed answer. It does not write prose, code, or explanations. TypeSafe AI, which started the category with Jev in September 2026, calls these System One models, after Daniel Kahneman's distinction between fast, intuitive System 1 thinking and slow, deliberate System 2 reasoning. "Decision model" is the label most of the press and most competitors use.

A decision model behaves like a function call backed by a large pretrained network: unstructured state goes in, a typed, probabilistic decision comes out, and ordinary code decides what to do next. Because the request fixes the answer space, the model cannot return an off-schema value, and there is no generated text to parse, validate, or retry. A valid answer can still be the wrong answer, so the application keeps ownership of permissions, business rules, and the final action.

In this part: Where they're used · Timeline and vendors · How they work · Architecting · Design boundaries · Evaluating · Comparison · Jev · OpenAI · Clef · pplx-decider · Strands Decider · GLiDE · Kev · Other implementations

Where decision models are used

Decision models sit inside software and agent loops wherever a program needs a fast, bounded judgment rather than a paragraph. Typical uses:

  • Routing and triage: sending a ticket, request, or task to plain code, a specialist LLM, a cheaper or stronger model tier, or a human.
  • Tool, skill, and model selection inside agent loops, and deciding whether a step is finished.
  • Action gating: checking whether a proposed tool call should proceed, needs confirmation, or should go back to the agent.
  • Guardrails and trust and safety: screening messages into and out of an LLM application against a policy rubric.
  • Verification: estimating whether an LLM's output contains a fabricated field or a wrong citation, and escalating only when a check fires.
  • Context curation: scoring retrieved passages or candidate tools before an expensive model call.
  • Grading and judging: scoring outputs against rubric criteria, alone or as the first stage of an LLM-as-judge cascade.
  • Bulk labeling: converting large text corpora into structured variables (one published study coded police crash narratives at the scale of a state database).
  • Visual classification with the models that accept images (Clef, pplx-decider, and OpenAI's Decisions API).

Decision models complement frontier LLMs rather than replace them. The generative model plans, reasons, and writes. The decision model makes the many small, frequent judgments around it at a fraction of the latency and cost.

A new architecture category

Decision models reflect a design choice about what the model predicts: a calibrated distribution over declared answers instead of a token sequence. That one choice changes speed, cost, and failure modes. It puts decision models alongside diffusion language models, looped models, and JEPA as a 2025 to 2026 challenge to plain next-token prediction as the default training objective.

The category moved from one product to a crowded field in about two weeks:

Date (2026) Event
September 15 TypeSafe AI releases Jev, the first commercial System One model, in early access.
Late September Open reimplementations appear within days, including Laya, Kev, SemIf (formerly OpenJev), AnyJev, Mapika's decider family, and DiffusionGemma "structured reads" in vLLM.
September 29 OpenAI announces a Decisions API built on GPT-6 Luna at DevDay, in limited preview.
September 30 Fastino releases GLiDE, a decision model that spends extra reasoning on uncertain questions.
October 1 Cloudflare releases Clef and Clef-flash; Perplexity launches its Decisions API with pplx-decider-v1-27b; AWS Strands Labs releases Strands Decider 2B.

As of October 2, 2026, the vendors with a decision-model offering are TypeSafe AI, OpenAI, Cloudflare, Perplexity, AWS, and Fastino. Google has not shipped a decision-model product, although its open DiffusionGemma model is the basis of one community route and of Cloudflare's early experiments. NVIDIA researchers co-authored the CLM-8B research model with Stanford. No decision-model product had been announced by Anthropic, Microsoft, Meta, Mistral, or xAI. Almost every open entrant, including Clef, pplx-decider, Strands Decider, and Kev, is built on Alibaba's Qwen models.

How decision models work

The request contract

A request has two parts: a state (a string, a JSON object, or an array, and for some models images) and a map of named questions. Each question declares its type, instructions, and the allowed answers. Every question is evaluated against the same state, and the response returns one typed answer per question under the name the caller chose.

TypeSafe's "System One" request shape has become a de facto standard. Clef, Kev, Perplexity's Decisions API, and many open models use the same field names (state, questions, type, instructions, criteria), so switching providers usually means changing only the endpoint and model name. OpenAI has not published a schema for its Decisions API.

An illustrative request and response (values are examples, not real output): 

{
  "model": "<model-id>",
  "state": "Checkout has been failing for every customer for the last hour.",
  "questions": {
    "urgent":   { "type": "noul",   "instructions": "Is this support request urgent?" },
    "team":     { "type": "choice", "instructions": "Which team should handle this request?",
                  "criteria": { "billing":   "Payments, invoices, and refunds",
                                "technical": "Outages, errors, and configuration",
                                "sales":     "Plans and upgrades" } },
    "severity": { "type": "score",  "instructions": "How severe is the customer impact?",
                  "criteria": ["No impact", "Minor", "Major", "Critical"] }
  }
}

{
  "answers": {
    "urgent":   { "type": "noul",   "noul": 0.97 },
    "team":     { "type": "choice", "choice": "technical", "confidence": 0.93,
                  "probabilities": { "billing": 0.03, "technical": 0.95, "sales": 0.02 } },
    "severity": { "type": "score",  "score": 2.70, "confidence": 0.81,
                  "probabilities": { "0": 0.00, "1": 0.04, "2": 0.22, "3": 0.74 } }
  }
}
Offering Endpoint or call
Jev (TypeSafe) POST https://api.typesafe.ai/v1/systemone; also OpenRouter and Vercel AI Gateway
OpenRouter (Jev and others) POST https://openrouter.ai/api/alpha/decisions
Clef and Clef-flash (Cloudflare) Workers AI binding env.AI.run("@cf/cloudflare/clef", ...) or REST /ai/run/@cf/cloudflare/clef
pplx-decider (Perplexity) POST https://api.perplexity.ai/v1/decisions
Kev (self-hosted) POST /v1/systemone on a local server; the TypeSafe SDK works with a base_url change
OpenAI Decisions API Not published as of October 2, 2026

Question primitives

The System One contract defines three question types. All three can be mixed in one request, and each question is evaluated in isolation against the same state.

Primitive Purpose Returns
Noul Decide whether a statement is true, or answer a yes/no question Probability (0 to 1) that the answer is yes
Choice Select one option from a defined list Chosen option, per-option probabilities, confidence
Score Rate the state against ordered rubric levels Probability-weighted score, per-level probabilities, confidence

Limits differ by provider. Jev, Kev, and Perplexity accept up to 255 options per Choice question. Clef accepts up to 64 questions per request and Perplexity up to 128. Perplexity allows up to 10 Score levels. For option sets larger than the ceiling, score candidates independently first, then make an explicit choice from a shortlist.

Why decision models are fast and cheap

  • No autoregressive decoding. A chat model spells out JSON keys, a label, and perhaps a confidence field one token at a time. A decision model runs a prefill-style forward pass and reads answer probabilities directly from internal representations, so the server only serializes numbers.
  • Shared state, parallel questions. The state is encoded once and each question runs as an isolated branch. Kev, for example, packs the state and every question into one sequence with a block-causal mask so each question sees the state but never a sibling question. Adding questions therefore barely changes latency, so a whole battery of checks can cost about as much as one.
  • Input-only pricing. Hosted decision models typically bill input tokens only. Jev costs $0.042 and Perplexity $0.04 per million input tokens, with output free.

Vendor speedup figures come from vendor-chosen workloads. TypeSafe's "193.6x faster, 444.6x cheaper" headline and OpenAI's 10x claim are both measured against LLM baselines the vendor selected.

Computational routes

Products that share an API can compute decisions in very different ways:

Route How it works Examples
Purpose-built model (undisclosed) New architecture plus calibration-focused RL training; internals not public Jev
Encoder with a decision head Bidirectional encoder (ModernBERT class) scores option positions in one batched pass Laya
Decoder with adapter and pointer head Open LLM backbone with LoRA adapters; a small head scores each option's hidden state against a decision token Kev, Strands Decider 2B, Clef (with an added routing head), JEV-27B
Logit readout from an existing LLM Restricted next-token probabilities from an unchanged model, optionally corrected for option order, label priors, and temperature AnyJev, SemIf
Diffusion canvas answer slots One read-only denoising step returns log probabilities for allowed tokens at fixed answer slots DiffusionGemma structured reads (vLLM)
Constrained general LLM A frontier model limited to a predefined answer set OpenAI Decisions API (GPT-6 Luna)
Decision model with adaptive reasoning Extra internal passes on uncertain questions Fastino GLiDE

Probabilities, confidence, and calibration

  • Probabilities are the core output: a distribution over declared options or levels, or a single yes probability for Noul.
  • Confidence for Choice and Score is derived from that distribution and summarizes how concentrated it is. It is not independent evidence. Kev, for example, computes it as (pmax − 1/K) / (1 − 1/K), and Perplexity notes that its confidence drops when the runner-up option is close.
  • Calibration (whether a 0.8 answer is right about 80% of the time) is the main quality claim in the category. TypeSafe attributes Jev's calibration to RLCD; Cloudflare adds a Brier-score loss; Kev trains with cross-entropy on labeled outcomes. Independent work finds that probabilities often rank errors well even when the numbers themselves need recalibration, and that thresholds fitted on one workload do not always transfer to another. Reordering the option list can also shift the probabilities.
  • Practical rule: treat confidence as a second signal alongside the answer, and fit thresholds on held-out data from your own workload.

Lineage

The ingredients are old. BERT made pretrained encoders with small task heads the standard pattern for classification. Label-conditioned zero-shot classification showed that natural-language label descriptions could stand in for a fixed label set. Temperature scaling and selective classification brought calibration and risk-coverage evaluation into deep learning practice, and schema-constrained generation (OpenAI's Structured Outputs) gave LLMs hard output formats. Decision models package these ideas as an application contract: declare the state, declare the allowed answers, receive probabilities, and let code decide.

Component What it does Best fit
Conventional classifier Maps inputs to a fixed label set learned in training Stable labels with plenty of training data
Schema-constrained LLM Generates text that must fit a declared structure Extraction and longer reasoning
Decision model Scores the options supplied with each question and returns their probabilities New or changing label sets, many fast judgments, thresholded automation

Architecting with decision models

Reference pipeline

In practice, a decision model sits at three points around a frontier model: before it, beside it, and after it.

Stage Component Role Side exit
1 Incoming request Entry point None
2 Decision model: route and screen Classify intent, risk, and confidence Easy or uncertain cases go to plain code or a human
3 Decision model: curate context Score retrieved passages; select tools or skills None
4 Frontier LLM Reason and generate None
5 Decision model: verify output Estimate the probability that something is wrong, per check Flagged outputs escalate to a bigger model or a human, then re-enter stage 4
6 Policy check, then act or respond Enforce permissions and business rules in code; deliver the result None

A useful mental model is a five-layer agent harness: an orchestrator that holds the loop, a generative model that plans and writes, a decision model that answers narrow questions, a policy layer that enforces permissions, and an action-and-audit layer. The decision layer should never quietly become the policy layer. A 0.97 probability that an action "looks safe" does not grant a permission the account did not already have.

Integration patterns

TypeSafe's cookbooks document these patterns for Jev. Because most decision models share the same API, the patterns apply to any compatible model.

1. Routing in front

Many requests never need an expensive LLM call. In the intent-routing pattern, the decision model sits in front of every handler as a fast, cheap classifier that decides whether a request goes to deterministic logic, a specialist LLM, or a human. TypeSafe's customer-service example has four intents:

  • One intent is handled by plain code with no LLM.
  • Two go to different specialist LLMs loaded with different context.
  • For complaints, a complexity score decides whether the case goes to an LLM or a person.

Confidence acts as a second signal alongside the answer. If the model isn't confident enough about the intent, the ticket goes to a human agent instead of being guessed at. The same approach can pick a model tier, sending easy cases to a small model and saving the most expensive model for the hard ones.

2. Curating context before the call

Frontier models get slower and more expensive as context grows, and agents with large tool catalogs spend tokens just deciding which tool to use. A decision model can do this triage cheaply, much as DeepSeek's lightning indexer filters tokens before expensive attention, but at the level of whole passages and tools. TypeSafe's cookbooks include two examples:

  • Scoring each retrieved RAG passage in one request and letting code decide which passages reach the answering model.
  • Picking at most one skill per agent turn from the 182 in Nous Research's Hermes catalog.

Limit to plan for: because of the 255-option ceiling, a large tool registry needs a two-stage shortlist.

3. Verifying output and escalating only when needed

This is probably the strongest way to pair a decision model with a frontier model. In TypeSafe's structured-data-extraction (SDE) cascade, a cheap model does the extraction, and the decision model runs a yes/no check on each field that returns the probability something is wrong. Only if a check fires does the request go to an expensive reasoning model.

The worked example shows why this matters. The small model's output passed JSON Schema validation but contained a fabricated field. A schema check can't catch that kind of error, but a semantic verifier can.

Two design choices are important:

  • Phrase each question so that "true" means escalate.
  • Escalate if any single check fires, rather than averaging the checks, so one confident red flag isn't diluted.

In TypeSafe's internal test on 100 prompts, the cascade got most of the top model's quality at a fraction of its cost. The same approach catches wrong or fabricated citations by checking quotes against the source document. See Cascades and escalation for independent results that temper this.

4. Guardrails on inputs and outputs

One request can screen every message into and out of an LLM application. Hazard probabilities and severity thresholds decide whether to pass, review, block, or reroute each message. Because questions are evaluated in parallel and independently, adding more checks barely changes response time.

Caveat: decision models do not treat state as hostile by default. Injected instructions, misleading framing, or text that argues for its own classification can shift the answer. Use a decision model as one layer of defense, not the only one. See Zero Trust and Security and adversarial robustness.

5. Small decisions inside agent loops

The frontier model handles planning and writing. The many small decisions at each step can go to a decision model at roughly 40 to 500 ms each, depending on the model and hosting: which tool to call, whether a step is finished, whether an output looks right, whether a proposed action should proceed. AWS's Strands Decider example places the model immediately before a tool call and uses the agent framework's intervention hooks to proceed, deny, ask for confirmation, or send the agent back for missing information.

The same split works for bulk data. A decision model can map-reduce over very large datasets to turn them into features, and the aggregated results can then go to an LLM for synthesis.

Design boundaries

Decision model documentation consistently favors breaking questions down. If a question requires extended reasoning or weighs several factors, ask each factor separately and combine the results in code. A side benefit is interpretability: when every judgment is a small, named question, you can see exactly which one drove a decision (see Explainable AI).

Known weak spots recur across the category. TypeSafe documents them for jev-1.13, and open models report similar ones:

  • Literal reading of instructions.
  • Tasks that involve extra levels of indirection.
  • Arithmetic, counting, and date comparison (Kev's day-precision date arithmetic stayed near chance across every training recipe its author tried).
  • Accuracy loss as irrelevant state grows.
  • Adversarial content and contradictory criteria.
  • Structural consistency across separately asked questions, since each question is answered in isolation.
  • Knowledge-heavy questions, where smaller open models depend mostly on their base model.

The workarounds follow directly:

  • Keep arithmetic in code.
  • Extract the parts of a date, then compare dates in code.
  • Filter the state down to what each question needs.
  • Use a generative model for anything that requires generating text.

That defines the dividing line between components:

Task type Best handled by
Narrow, well-scoped judgments (classify, route, score, verify, gate) Decision model
Arithmetic, counting, date comparison, schema validation, permissions Application code
Text generation, multi-step or multi-hop reasoning Frontier LLM or reasoning model
Low-confidence or high-stakes decisions Escalation to a stronger model or a human

Evaluating decision models

Coverage and accepted-case error

A decision model in production accepts some answers and defers the rest to a fallback. Two numbers describe that policy, and they use different denominators:

  • Coverage: the share of all cases the model handles itself (for example, 60 of 100).
  • Accepted-case error: the share of accepted answers that are wrong (for example, 3 of those 60, or 5%).

Raising the confidence threshold lowers both. Report raw counts for both, and fit the threshold on data separate from the final evaluation, grouped by whatever unit could leak (a customer or a document). Kev reports 45% to 57% coverage at a 5% accepted-case error limit across its checkpoints, against 70% for Jev on the same items. Escalation logs only contain the cases the model set aside, so once traffic is live, sample accepted answers to find confident mistakes.

Cascades and escalation

A cascade beats its own fallback only when the decision model is more accurate than the fallback on the cases it accepts. Two independent September 2026 studies tested this:

  • JEV-as-a-Judge (510 held-out preference pairs): at a 0.90 threshold with GPT-6 Astra as fallback, Jev handled 53.7% of pairs. The cascade reached 92.5% accuracy against 93.1% for GPT-6 Astra alone, at roughly 57% to 62% of the fallback's estimated fees.
  • Jev vs. LLMs as Rubric Judges (5,003 submission-criterion pairs): Jev's confidence ranked its own errors on most panels, but the fallback LLM judges repeated Jev's high-confidence errors in 96% of sampled verdicts, so escalation added little accuracy. Several cascades still cut cost.

The lesson: a cascade's cost savings and its accuracy have to be measured separately, and escalation cannot fix errors the fallback model shares.

Benchmarks and leaderboards

Most published comparisons are vendor-run, use vendor-chosen tasks, and are not directly comparable to each other.

  • JevBench (open project; v1.5.4 had 1,624 decisions and 112 configurations as of October 2, 2026) combines intelligence, calibration, speed, and cost into one composite index. Its top three were Cygnet (blockbrain, on a frozen Gemma-4-12B), Winnow-12B, and Jev 1.13.0. General LLM baselines such as GPT-6 Luna score far higher on the intelligence component (about 95 against 72 for Jev) but rank lower overall because the composite also weighs speed and cost.
  • Jev Decision Index (community project on Hugging Face) benchmarks open reproductions of Jev. Cloudflare and Fastino both report results on it.
  • Vendor panels: Cloudflare's 10-benchmark table, Perplexity's 11-task panel, AWS's JevBench public-set results, and TypeSafe's private workflow evals. Each is summarized under its offering below.
  • Rule of thumb: benchmark the model on your own workload before building around it.

Security and adversarial robustness

A typed output limits which decisions a model can return, but not how untrusted input influences which one it picks.

  • Decision hijacking (510 reconstructed InjecAgent cases): malicious content shifted Jev's action probabilities but rarely made it select the attacker's target action.
  • JevOut: ordinary, plausible added context that leaves the correct answer unchanged can still flip a decision model's choice, and contexts crafted against one model transferred to other decision models.
  • Practical rule: a cheap probabilistic guard is not a security boundary. Pair it with allowlists, permission checks, and human review for actions that touch money, access, or safety.

Comparison of offerings

Figures are as published by each vendor between September 15 and October 2, 2026. Check vendor pages for changes.

Offering Maker Released Access Size and base Input Context Hosted price per million input tokens Weights
Jev 1.13 TypeSafe AI Sep 15, 2026 Hosted API (TypeSafe, OpenRouter, Vercel AI Gateway) Not disclosed Text, JSON 32K on OpenRouter; 64K per TypeSafe docs $0.042, output free Closed
Decisions API OpenAI Announced Sep 29, 2026 Limited preview Constrained GPT-6 Luna Text, images Not published Not published Closed
Clef Cloudflare Oct 1, 2026 Workers AI; self-host 27B (frozen Qwen3.8-27B + adapters) Text, JSON, up to 4 images 64K $0.24 Apache 2.0
Clef-flash Cloudflare Oct 1, 2026 Workers AI; self-host 9B (frozen Qwen3.5-9B + adapters) Text, JSON, up to 4 images 64K $0.09 Apache 2.0
pplx-decider-v1-27b Perplexity Oct 1, 2026 Decisions API; self-host 27B (Qwen3.8-27B fine-tune) Text, JSON, images Under 262,144 tokens per request $0.04, output free Apache 2.0
Strands Decider 2B AWS Strands Labs Oct 1, 2026 Self-host only About 2B (Qwen3.5-2B + LoRA + pointer head) Text Not stated None (no hosted API) Apache 2.0, with training data
GLiDE Fastino Sep 30, 2026 Hosted API Not disclosed Not stated 40,000 tokens per rendered question $0.30, output free Not stated
Kev Jared Palmer (independent) Sep 20, 2026 (Qwen3.5 generation) Self-host; Kev 4B also on OpenRouter 0.8B, 4B, 9B (Qwen3.5); 27B (Qwen3.8-27B) Text, JSON 8,192 tokens serving cap Free to self-host; Kev 4B $0.042 on OpenRouter Apache 2.0 adapters and heads

Clef prices are from Cloudflare's Workers AI pricing as reported by CellCog on October 2, 2026. For Kev, the base Qwen weights carry the Qwen license.

Jev (TypeSafe AI)

Jev is the first System One model from TypeSafe AI, released in early access on September 15, 2026. It is closed and hosted, text-only, and the reference point every other offering on this page compares itself to.

Company

TypeSafe was founded by Diogo Almeida, a former OpenAI researcher who worked on the instruction-following methods behind ChatGPT, along with co-founders Erik Gafni and Sasha Sheng. The company came out of stealth with a $40 million seed round led by DCVC. The model's name honors the economist William Stanley Jevons: TypeSafe expects cheaper intelligence to increase total demand for it, the way more efficient steam engines increased demand for coal.

How Jev works

TypeSafe says it built three new pieces:

  • A new model architecture designed for decisions rather than strings.
  • A parallel sampler that evaluates every question in a request at once instead of one token at a time. That is the main reason it's fast.
  • RLCD (Reinforcement Learning for Calibrated Decisions), a training method that rewards honest probabilities rather than answers humans prefer. TypeSafe argues that human-preference training (RLHF) causes mode dropping and overconfidence, which is why LLMs need humans in the loop.

TypeSafe says the model was trained only on synthetic data. It has not disclosed the parameter count, weights, or full architecture. A widely shared post by researcher Niels Rogge, generated by having a coding agent search the research literature, guessed that Jev is a large schema-conditioned bidirectional encoder with parallel label-query heads trained with a proper-scoring-rule RL objective, and that the speedup comes from removing autoregression rather than from any exotic mechanism. Kev follows a different public reconstruction (Archer Hume's "Jev's Architecture Unmasked"). Treat both as informed speculation, not confirmed fact.

Access and pricing

  • Model IDs: jev-1.13.0 (pinned, for example typesafe/jev-1.13-20260917 on OpenRouter) and jev-latest.
  • Channels: TypeSafe's API and SDKs (Python and JavaScript), OpenRouter's Decisions API, Vercel AI Gateway, and integrations including Netlify and OpenClaw's decision-model role.
  • Pricing: $0.042 per million input tokens, with free output tokens.
  • Latency: TypeSafe quotes 70 to 500 ms end to end. OpenRouter's production telemetry showed a P50 of 0.21 s and P95 of 0.34 s on October 1, 2026. Cloudflare measured a 524 ms median in its own benchmark runs.
  • Context: 32K tokens on OpenRouter; TypeSafe's docs cite 64K per request.
  • Choice limit: up to 255 options; for larger sets, TypeSafe scores candidates independently first and then makes an explicit choice.

Known limitations (jev-1.13)

TypeSafe's jaggedness page lists literal reading, weak indirection, weak arithmetic and date comparison, sensitivity to long irrelevant state and adversarial content, contradictory criteria, and inconsistency across separately asked questions. See Design boundaries for the workarounds, which apply to every model on this page.

Adoption and reception

  • Adoption: Jev reached Vercel's AI Gateway the day after launch. Vercel reported that nearly 13% of paid teams used it within 24 hours, about twice as many as any earlier model launch, making it the gateway's fastest-adopted model. Almeida has said traffic passed one trillion tokens per day.
  • Headline claims: TypeSafe reports Jev as "193.6x faster, 444.6x cheaper" than LLMs on its own workflow evals and says those figures are likely at the high end of real-world gains. Those evals use the average of GPT-6 Astra and Fable 5.1 as the reference answer, which TypeSafe acknowledges biases results toward those model families. The French outlet Next noted that head-to-head comparisons with LLMs remain tricky.
  • Benchmarks: TypeSafe rejects public benchmarks in favor of private evaluation. On the independent JevBench 1.5.4 composite, Jev 1.13.0 ranked third as of October 2, 2026. Almeida also argues that safety refusals behave like type errors when a model is buried inside software.
  • Criticism: In a long livestream review, developer Theo Browne argued that Jev is best understood as a System 1 classifier and that many viral demos (context compaction, LLM-as-judge, code review) misuse it. Hacker News commenters noted that its probabilities shift when the option list is reordered.
  • Competition: Within two weeks of launch, OpenAI, Cloudflare, Perplexity, AWS, and Fastino all shipped rivals, and several open models published head-to-head numbers against Jev. Jev still leads or ties on several vendor tables, including When2Call and BRIGHT in Cloudflare's own results.

Jev sources

OpenAI Decisions API

OpenAI announced the Decisions API at DevDay on September 29, 2026, two weeks after Jev's launch. Simon Willison, live-blogging from the event, described it as sounding like a response to Jev.

How it works

Unlike Jev, the Decisions API is not a separate model. OpenAI describes it as focusing a version of GPT-6 Luna, the smallest and cheapest model in the GPT-6 family, on developer-defined questions with a finite set of predefined answers. Developers supply context as text or images and get back an answer to classify content, route requests, or choose an agent's next action. Sam Altman demonstrated it on stage driving a computer-use agent in real time.

Status and claims

  • Access: limited preview for selected API customers, with a broad release described as planned "in the coming days." It was still in limited preview as of October 2, 2026.
  • Latency: OpenAI's DevDay slide claims about 150 ms per decision against 1.6 s for a standard GPT-6 Luna call, a 10x speedup measured against OpenAI's own baseline.
  • Not yet published: the endpoint path, request and response schema, SDK methods, whether probabilities or confidence scores are returned, multi-question requests, context limits, and pricing. Plain GPT-6 Luna lists at $0.10 input and $0.50 output per million tokens, but OpenAI has not said whether the Decisions API uses those rates.
  • Name collision: OpenRouter's earlier "Decisions API" (/api/alpha/decisions) is a different product that serves Jev and other decision models.

Positioning

The Decisions API is the clearest example of the constrained general LLM route: one model that can still generate text, narrowed to a decision job. Its main advantage over Jev is image input. Until OpenAI publishes a contract, any example request body for it is invented, so integrate only behind a thin adapter.

OpenAI Decisions API sources

Clef and Clef-flash (Cloudflare)

Clef and Clef-flash are the first models trained by Cloudflare's Workers AI team, released October 1, 2026. Both are fully Jev-API compatible, hosted on Workers AI, and open-sourced on Hugging Face under the Apache 2.0 license, so they can also be run locally. Cloudflare named the family after the musical clef, which fixes the pitch of the notes that follow it, as a decision fixes the actions that follow it.

Clef models

Model Size and base Best for Context Median / p95 latency (Cloudflare's runs)
@cf/cloudflare/clef 27B, frozen Qwen3.8-27B plus adapters Highest-precision decisions 64K tokens 209.3 ms / 238.6 ms
@cf/cloudflare/clef-flash 9B, frozen Qwen3.5-9B plus adapters Latency-critical, hot-path decisions 64K tokens 38.8 ms / 122.4 ms

For comparison, Jev measured 524.1 ms median in the same runs. Part of the latency gap comes from hosting on Cloudflare's network close to the caller.

What sets Clef apart

  • Vision: a vision encoder lets callers pass up to four images alongside the state. The model card also mentions video frames.
  • Context: 64K tokens, against 32K for Jev on most gateways.
  • Up to 64 questions per request in the standard Noul, Choice, and Score types.
  • Data handling: Cloudflare states that it does not read, store, or train on requests or responses, except for customers who opt into fine-tuning.

How Clef was trained

Clef grew out of a Cloudflare experiment that exposed DiffusionGemma's log probabilities as decisions, building on Matt Mastracci's vLLM work. The released models switch to Qwen backbones. At inference, Clef runs a prefill-only pass and scores the valid schema choices in parallel through a two-stage attention routing head, so each option gathers evidence from the payload and fields can attend to each other before scoring. Cloudflare froze the Qwen backbones and jointly trained the routing head with rank-256 LoRA adapters, using label-smoothed cross-entropy plus a Brier loss for calibration on internal synthetic data that permutes field orders, prompts, and schemas. Its own version of RLCD serves as a secondary objective that gives partial credit to adjacent ordinal choices and penalizes drift from a reference model.

Benchmarks (Cloudflare-run)

  • Decision benchmarks: a Clef model scores highest on 7 of 10. Highlights include BANKING77 macro-F1 of 94.20 for Clef against 79.74 for Jev, and Home appliances case-exact accuracy of 97.73 for Clef-flash against 52.27. Jev still wins When2Call and BRIGHT, a DiffusionGemma-based model wins PhishNChips, and Clef-flash drops sharply on CLINC150+OOS.
  • TypeSafe's own workflow evals: Clef beats Jev on invoice processing, customer service, and security incidents; Jev wins agent trace observability.
  • In production at Cloudflare: the Threat Intelligence team paired Clef with Browser Run to fetch, render, and classify a domain in 2.2 seconds, against 4.7 seconds for gpt-oss-120b in the same workflow.

Cloudflare chose the tasks and ran every model, and no independent comparison covering all models exists yet.

Fine-tuning service

Cloudflare launched a reinforcement-learning fine-tuning service alongside the models. It starts as a hands-on engagement with Cloudflare's forward-deployed engineers and is planned to become self-serve: AI Gateway captures request data, Workers AI generates rollouts, Containers host RL sandboxes, a new Trainer updates weights, and the tuned model is redeployed on Workers AI. Cloudflare is recruiting design partners.

Pricing and self-hosting

Hosted pricing is about $0.24 per million input tokens for Clef and $0.09 for Clef-flash. The model card loads the weights with Hugging Face Transformers and PyTorch, and Cloudflare tested local inference on a single NVIDIA H200.

Clef sources

pplx-decider (Perplexity)

pplx-decider-v1-27b is Perplexity's multimodal decision model, fine-tuned from Qwen3.8-27B and released October 1, 2026. It powers Perplexity's Decisions API, and its weights are on Hugging Face with an Apache 2.0 license tag. The Hugging Face repository notes that it was duplicated from denis-pplx/autojev-27b, an earlier checkpoint also listed on community leaderboards.

Decisions API

  • Endpoint: POST https://api.perplexity.ai/v1/decisions, using the same state and questions shape and the same noul, choice, and score types as Jev. Any Perplexity API key works.
  • Input: text, JSON, or images. Images must be PNG, JPEG, or WebP data URLs (the API never fetches remote URLs) and cost about 1,000 input tokens per megapixel.
  • Limits: 1 to 128 questions per request, up to 255 Choice options, up to 10 Score levels, under 262,144 input tokens per request, and a 32 MiB body. Each organization can send 10 requests per second.
  • Latency: grows with input size. In Perplexity's tests, a few hundred input tokens answered in under 2 seconds, about 90,000 tokens in 5 seconds, and about 190,000 tokens in 14 seconds.
  • Pricing: $0.04 per million input tokens, with free output and no per-request fee, slightly under Jev.
  • Behavior notes: identical requests occasionally differ in the second decimal place, so set thresholds with some margin. Perplexity's suggested pattern is to triage tickets with the Decisions API and send only escalations to its Agent API.

Benchmarks (Perplexity-run)

On a fixed 11-task, 7,210-row panel, Perplexity reports 85.71% overall accuracy for pplx-decider-v1-27b, 84.51% for Jev 1.13.0, and 74.76% for the base Qwen3.8-27B. The lead is not uniform: Jev scores higher on 6 of the 11 tasks (WinoGrande, JudgeBench, BBH, JevBench public hard, Belebele, and TruthfulQA binary), while pplx-decider leads clearly on RAGTruth (88.80% against 77.27%) and FinancialPhraseBank (84.18% against 76.98%). These are self-reported results, and task sizes differ, so the overall figure weights them unevenly.

pplx-decider sources

Strands Decider 2B (AWS)

Strands Decider 2B is an open decision model released October 1, 2026 by Strands Labs, AWS's experimental agent-development project. AWS's pitch is not that it beats Jev, but that it is small, local, and fully reproducible.

Architecture

AWS started from the pretrained Qwen3.5-2B base, removed the next-token prediction head, and replaced it with a small pointer component (about one million parameters, in a design AWS labels "Hobson") that scores the supplied answer options. The backbone is adapted with a rank-16 LoRA update. The model makes one forward pass and returns a distribution over the available choices.

Access

  • Weights: on Hugging Face (StrandsAgents/strands-decider-2B-hobson-v19) under Apache 2.0.
  • Openness: AWS says version 20 will ship with the code, training data, and scripts, and it published the trajectory of every version it tried.
  • Hosting: self-hosted only, with no hosted API or per-call fee. It runs on a laptop or a single GPU.
  • Framework integration: works with the Strands Agents intervention mechanism, which can proceed, deny a tool call, request confirmation, or guide the agent with feedback. AWS's example checks whether a weather-tool call guessed a city the user never named, then sends the agent back to ask.

Performance (AWS-reported)

  • Latency: version 18 recorded a 106 ms median and 296 ms p95 across 230 requests on a local NVIDIA RTX 3090, including the HTTP round trip, and about 150 ms median for small tasks on an M3 MacBook. Latency rises roughly linearly with input size.
  • Quality: version 19 answered 167 of 231 items on the public JevBench set (about 72%) with a Brier score near 0.35, and cleared every item in the easy tier. Mapika's similarly sized decider-2b scored better on both measures (about 76% and 0.32). AWS's chart does not include Jev.

Strands Decider sources

GLiDE (Fastino)

GLiDE is a hosted decision model released by Fastino on September 30, 2026. It returns Noul, Choice, and Score answers with probabilities and confidence, like the rest of the category, but it spends additional reasoning on uncertain decisions, which Fastino presents as the first "thinking" decision model. Fastino also makes the GLiNER family of small extraction and classification models.

  • Limits and pricing: the API accepts 40,000 tokens per rendered question prompt. Fastino lists $0.30 per million input tokens with free output, and input usage sums the internal passes across questions, so uncertain questions cost more.
  • Benchmarks (Fastino-reported): 64.81 skill points on Decision Index 0.2.1 against 57.91 for Jev 1.13.0, with the largest margins on CLadder causal reasoning (88.7% against 72.6%) and CRUXEval code reasoning (92.6% against 73.0%). GLiDE is not on the public Decision Index board, and Fastino published exact values for only eight metrics.
  • Trade-off: adaptive reasoning targets the category's weak spot (multi-step judgments), at about seven times Jev's input price.

GLiDE sources

Kev (Jared Palmer)

Kev is a family of open-source decision models by developer Jared Palmer, built with help from the Devin coding agent. It uses the same API as Jev, so applications built with TypeSafe's SDK can switch by changing the endpoint and model name. The model only scores supplied options and doesn't generate explanations or retrieve missing facts. The repository says no Jev outputs were used in training.

Kev checkpoints

Checkpoint Base Notes
Kev-0.8B Qwen3.5-0.8B-Base Runs on any Apple Silicon Mac
Kev-4B Qwen3.5-4B-Base Recommended starting point; serves on a 32 GB Mac in bf16; also hosted on OpenRouter
Kev-9B Qwen3.5-9B-Base For a bigger GPU
Kev-27B Post-trained Qwen3.8-27B Most accurate; sized for a single 80 GB GPU

The Qwen3.5 generation (0.8B, 4B, 9B) was released September 20, 2026, replacing earlier Qwen2.5 and Qwen3 research previews (0.5B, 0.6B, 4B, 8B). The Qwen3.5 port cost roughly $95 of rented H100 time.

How Kev works

Kev is a rank-16 LoRA adapter plus a small pointer head on a Qwen base. It packs the state and every question into one sequence. A block-causal mask lets each question see the state but never another question, so question order does not matter and packed requests match separate ones to within a few millionths. The pointer head compares the hidden state of a closing decision token with each option's closing token and applies a softmax. Training uses cross-entropy against labeled outcomes from about ten public datasets plus generated policy and rule examples.

Results

  • On the repository's out-of-domain suite (sources Kev never trained on), Kev-27B scores 0.848 on the development split against 0.857 for Jev, with Kev-4B and Kev-9B within about four points. On the locked test set, Kev-9B scored 0.837.
  • At a 5% accepted-case error limit, Kev's checkpoints keep 45% to 57% coverage against 70% for Jev.
  • Kev trails Jev most on knowledge questions (MMLU 0.84 for Kev-27B against 0.90), paraphrase, and day-precision date arithmetic.
  • The Qwen3.5 models are slower than the Qwen3 previews on Apple hardware (779 ms for five questions on Kev-4B on an M5, against 174 ms before).

Limitations

Calibration holds in-distribution only; temperature fitted in-domain does not transfer. The local server handles one request at a time and caps context at 8,192 tokens. The code, weights, and evaluation reports are available, but the full 27B training corpus and some evaluation data are private.

Kev sources

Other open and research implementations

Dozens of open reimplementations appeared within two weeks of Jev's launch. They share the "decision model" label but differ in architecture, training, and how well their probabilities hold up, so their benchmarks are not interchangeable.

Project Maker Approach Notes
Laya Convai Innovations (Nandakishor M) ModernBERT-large encoder (421M) or multilingual mmBERT (322M) with a trained decision head; one batched pass Extremely fast (5.8 ms median in Cloudflare's runs) but low accuracy on general benchmarks; Apache 2.0
AnyJev Nokia Applied Research Readouts from an unchanged open decoder: restricted logits, option-order and prior correction, temperature fitting, or a closed-form head fit from 100 to 300 labeled examples On BANKING77 with Qwen3-8B, coverage at 5% error rose from 7.7% (raw logits) to 52.0% after correction
SemIf (formerly OpenJev) Theodore Lee Reads option logits off a frozen Qwen3.5-4B, no fine-tuning About 5x faster than generating JSON on the same 21 decisions; MIT license
DiffusionGemma structured reads vLLM contributors; model by Google DeepMind One read-only denoising step over fixed answer slots in a seeded diffusion canvas Merged into vLLM September 22, 2026; DiffusionGemma is an open Apache 2.0 diffusion LM with about 26B total and under 4B active parameters
CLM-8B Stanford and NVIDIA researchers Caches representations of reusable agent actions Up to 9x faster than Jev on some tasks in the researchers' tests, but trails on tool-calling accuracy
JEV-27B AutoTrust AI 108.9M-parameter decision block (about 0.4% of the model) on a frozen Qwen3.8-27B; the base model's generation path is untouched Vendor reports 96% of Jev 1.13's accuracy on one independent benchmark; Apache 2.0
decider family Mapika Fine-tuned decision models at 2B, 4B, and 35B-A3B with published training code decider-4b v2 ranked seventh on JevBench 1.5.4
Cygnet blockbrain Recipe on a frozen Gemma-4-12B Joint leader of JevBench 1.5.4 as of October 2, 2026
Bespoke Nimble 9B Bespoke Labs Apache-licensed adapter with reference code Listed on JevBench

Other implementation sources

Ecosystem Fit

AI architecture sits underneath nearly every other part of the AI ecosystem.

Model Layer

At the model layer, architecture determines how information is represented and processed. Examples include:

  • Transformers (increasingly with gated and sparse attention).
  • Mixture-of-Experts (MoE).
  • State-space and linear-attention hybrids.
  • Diffusion language models.
  • Reasoning models built on dense or MoE bases.
  • Looped language models.
  • Graph neural networks.
  • Convolutional networks.
  • Joint Embedding Predictive Architectures.
  • Generative world models.
  • Recursive reasoning models.
  • System One decision models (see Decision Models).
  • Ensembles.
  • Neuromorphic systems.
  • Hybrid systems.

These approaches aren't necessarily competitors. A single production system can, and often does, combine several of them.

Generative AI

Generative AI Stack systems usually depend on transformer-based architectures, but the surrounding stack also includes:

  • Data pipelines.
  • Embeddings.
  • Retrieval.
  • Memory.
  • Model serving.
  • Tools.
  • APIs.
  • Routers, verifiers, and guardrails, increasingly built on fast decision models.
  • Safety systems.
  • User interfaces.
  • Compute infrastructure.

Model architecture is one layer inside a much larger production architecture. In 2026, a growing share of the engineering effort goes into the layers around the model. Teams spend that effort deciding what reaches the model, checking what comes out, choosing when a cheaper component can do the job, and deciding when a reasoning model's extra thinking time is worth paying for.

Agents and World Models

Architectures such as JEPA become especially interesting when AI moves from generating answers to understanding environments and choosing actions.

A world model tries to answer one question:

If the world is in this state now, what's likely to happen next, especially if I take a particular action?

That capability connects prediction to planning, and it depends on learning genuine cause and effect rather than surface patterns (see Causation vs. Correlation).

Two camps now compete to build these models:

  • JEPA-style systems, backed by Meta FAIR and AMI Labs, predict abstract future states and plan in that latent space.
  • Generative systems, such as Google DeepMind's Genie 3 and World Labs' Marble, render the future in full detail and let agents act inside the simulation. NVIDIA offers its own open-weight world models, called Cosmos.

Hardware makers are now buying in directly: AMD's agreement to acquire World Labs means generated worlds may soon ship alongside the chips that train robots inside them. Neither camp has matched humans on intuitive-physics benchmarks yet. Both are aimed at the same targets:

  • Robotics.
  • Autonomous systems.
  • Physical AI.
  • Simulation and synthetic training environments.
  • Interactive environments.
  • Long-horizon planning.

Memory and Lifelong Learning matter here too, because these systems need to accumulate experience over time instead of restarting from a fixed pretrained state. That's exactly the gap that continual-learning architectures like Hope are trying to close.

Enterprise Architecture

Enterprise Architecture (EA) operates above the individual AI model. Enterprise systems must decide:

  • Which models to use, and which tasks can go to cheaper specialized models such as decision models.
  • When to pay for a reasoning model's extra inference time.
  • Where models run.
  • How data reaches them.
  • How models access business systems.
  • How permissions are enforced.
  • How services communicate.
  • How models are monitored and their outputs verified.
  • How systems recover from failure.
  • How AI fits existing business processes.

This connects AI model architecture with Service Capabilities, Messaging & Routing, Enterprise Portfolio Management (EPM), and Zero Trust.

Compute Infrastructure

Modern AI architectures are tightly coupled to their hardware. Processing Units - CPU, GPU, APU, TPU, VPU, FPGA, QPU support different workload characteristics, and large models may spread computation across hundreds or thousands of accelerators.

That requires architecture for:

  • High-speed networking.
  • Memory management, especially KV caches for long context and long reasoning traces.
  • Distributed training.
  • Distributed inference.
  • Model partitioning, including expert placement for MoE models.
  • Low-precision formats (such as FP8 and NVFP4) to cut memory and bandwidth.
  • Numerical stability at low precision, which fixes like gated attention and QK-Clip help protect.
  • Storage.
  • Fault tolerance.
  • Scheduling.

Model architecture and compute architecture now have to be designed together. Several recent designs exist largely because of hardware limits on memory bandwidth and energy:

  • Sparse experts.
  • Gated and sparse attention.
  • Hybrid attention.
  • Hardware-aware recurrences like Mamba-3's MIMO formulation.
  • Parallel decoding.

The co-design is also showing up in corporate strategy, with chipmakers investing in and now acquiring model companies.

Cloud, Containers, and Edge

Production AI may run across several environments:

  • Centralized cloud infrastructure.
  • Private data centers.
  • Edge servers.
  • Personal computers.
  • Smartphones.
  • Vehicles.
  • Robots.
  • Embedded systems.

Containers; Docker, Kubernetes & Microservices let individual AI services be packaged and scaled independently.

Serverless architectures can handle event-driven AI functions without dedicated application servers.

Edge architectures reduce latency and keep sensitive data closer to the user, but they have to work within tighter compute, memory, and energy limits. Several newer designs are attractive at the edge for exactly that reason:

  • Small MoE models with few active parameters.
  • Looped models that get more depth from fewer weights.
  • State-space hybrids with flat memory use.
  • Neuromorphic chips.

Sociotechnical Systems

AI architecture doesn't stop at the technical boundary. Real systems contain:

  • People.
  • Organizations.
  • Policies.
  • Models.
  • Data.
  • Interfaces.
  • Business processes.
  • Incentives.
  • Regulations.
  • Security controls.

These are sociotechnical systems'. An AI system can be technically sophisticated and still fail if its architecture doesn't account for how people actually use it. Confidence-gated designs, where uncertain cases go to a human (see Architecting with decision models), are one way to build that awareness directly into the architecture. Independent peer review of model claims is another. Natures editors argued in 2025 that it helps dial back hype in the AI industry.

Architecture has to answer both:

How does the technology work?

and

How does the technology fit into the larger human system?

Will AGI Require New Architectures?

Whether artificial general intelligence will require fundamentally new architectures remains an open research question, and it's now being tested with real money. AMI Labs raised over a billion dollars on the bet that JEPA-style world models, not scaled-up language prediction, are the path forward. AMD's $8.2 billion agreement for World Labs is an equally large bet on generative spatial intelligence.

The evidence cuts both ways:

  • Extend what we have. Increasingly capable systems may emerge from today's architectures with more compute, better data, longer context, improved memory, stronger tool use, multimodal perception, and better learning methods. DeepSeek-R1's reasoning gains, which came from a new training stage on an existing MoE architecture, support that view. So does Kimi K2, whose biggest leap was an optimizer.
  • Change the architecture. Key capabilities, such as physical common sense and continual learning, may require new designs. The persistent human-versus-model gap on intuitive-physics benchmarks supports that view.
  • Somewhere in between. Looped models like Ouro add a third scaling axis (reusing layers) without abandoning the transformer at all.

Current research explores several directions:

  • Mixture-of-experts: increasing capacity while activating only part of the model.
  • Sparse and gated attention: spending attention only where it matters, and stably.
  • Hybrid architectures: combining transformers with state-space, linear-attention, and other computational approaches.
  • Parallel decoding: generating many tokens at once through diffusion instead of one at a time.
  • Reinforcement-learned reasoning: training models to think longer and check their own work.
  • Looped and recursive architectures: getting depth from repetition rather than more parameters.
  • Hierarchical architectures: reasoning and planning across multiple levels of abstraction.
  • World models: learning how environments change so an AI can predict and plan, either in latent space (JEPA) or by generating the world (Genie 3, Marble).
  • Continual-learning memory: multi-speed memory systems that let models keep learning after deployment.
  • Calibrated decision models: separating fast, reliable judgments from slow, open-ended reasoning.
  • Neuromorphic architectures: exploring event-driven and biologically inspired computation.
  • Collective and ensemble systems: allowing several models or agents to work together.
  • Persistent memory: allowing systems to accumulate and reuse information over long periods (see Memory Networks).
  • Embodied architectures: connecting AI directly with sensors, actions, robotics, and physical environments.

It's too early to know whether any one of these directions will dominate. The most likely near-term pattern is architectural convergence: multiple specialized approaches working together inside increasingly complex AI systems, with capabilities that emerge from the combination.

Additional Viewing

These long-form videos (each longer than 20 minutes) cover architectures discussed on this page.

Transformer Architecture

Load video
YouTube

Stanford CME295 Transformers & LLMs, Lecture 1: Transformer (Stanford Online, Autumn 2025). This graduate lecture builds the transformer from the ground up, moving from tokenization and embeddings through recurrent networks to self-attention and the full architecture. It's the clearest way to understand why attention replaced recurrence and what problem each transformer component solves.

Large Language Models and Mixture-of-Experts

Load video
YouTube

Stanford CME295 Transformers & LLMs, Lecture 3: Transformers & Large Language Models (Stanford Online, Autumn 2025). The lecture moves from basic transformers to modern LLM design, with a detailed comparison of dense versus sparse Mixture-of-Experts (MoE) architectures. It also covers the inference machinery that decides real-world cost, including key-value caching, PagedAttention, and multi-head latent attention.

Attention Alternatives: Linear Attention and MoE

Load video
YouTube

Stanford CS336 Language Modeling from Scratch, Spring 2026, Lecture 4: Attention Alternatives (Stanford Online, April 2026). The lecture explains why attention's cost grows with the square of sequence length, and why FlashAttention gives a large constant-factor speedup without changing that scaling. It then shows how linear attention reorders the math to reach linear cost and RNN-like inference, and presents mixture-of-experts as the matching strategy for scaling the feed-forward layers.

JEPA, Self-Supervised Learning, and World Models

Load video
YouTube

Yann LeCun: Self-Supervised Learning, JEPA, World Models, and the Future of AI (Harvard Center of Mathematical Sciences and Applications, 2025). Yann LeCun argues that generative next-token prediction can't produce systems that understand the physical world, and makes the case for predicting in representation space instead. He connects self-supervised learning, joint embeddings, world models, and planning into the research agenda that later became AMI Labs.

Load video
YouTube

Stanford CS25: Transformers United V6, From Representation Learning to World Modeling (Stanford Online, recorded April 9, 2026). Brown University researchers Hazel Nam and Lucas Maes (lead author of LeWorldModel) argue that world models are shifting away from reconstructing their inputs and toward predicting in latent space. They illustrate that shift with two recent JEPA-based approaches that attack the problem from complementary angles.

Recursive Reasoning Models

Load video
YouTube

Recursion Is The Next Scaling Law In AI (Y Combinator, May 2026, about 37 minutes). YC's Ankit Gupta and Francois Chaubard walk through the Hierarchical Reasoning Model and Tiny Recursive Model papers, and explain why standard LLMs hit a depth ceiling on some algorithmic tasks. Their central argument is that an outer refinement loop at inference time gives small models "compute depth" without parameter depth, and that combining this with large foundation models is a promising hybrid.

System One Decision Models and Jev

Load video
YouTube

Why We Made Jev: Diogo Almeida, TypeSafe Co-founder & CEO (Latent Space, September 2026, about 2 hours 21 minutes). Host swyx interviews Diogo Almeida days after Jev's launch about why chat-optimized models struggle to automate basic work, and why he built models for code instead of people. They cover RLHF mode collapse and calibration, refusals as type errors, TypeSafe's rejection of public benchmarks, and the design of Jev's Choice, Score, and Noul primitives.


Sources and Further Reading

Annotated reading lists by topic. Sources for individual decision-model offerings (Jev, OpenAI's Decisions API, Clef, pplx-decider, Strands Decider, GLiDE, Kev, and open implementations) are listed under each offering in the Decision Models part. Cross-cutting decision-model research is in Decision Model Research and Comparisons below.

Mixture-of-Experts and Attention

Sparse Attention and Training Stability

Hybrid Sequence Models and Diffusion

Reasoning, Memory, and Continual Learning

JEPA and World Models

Recursive and Looped Reasoning

Decision Model Research and Comparisons

Neuromorphic Computing

Retrieved from "https://primo.ai/w/index.php?title=Architectures&oldid=46276"