On September 4, 2026, Gimlet Labs said it had raised $300 million in a Series B led by Andreessen Horowitz, at a $3 billion valuation, to build what it calls a multi-silicon inference cloud for agentic AI. The number is large, but the more interesting part is the thesis behind it: that the cheapest and fastest way to run a model is not to pick the best chip, but to stop running the whole model on one chip at all.
That claim only makes sense once you look at what an inference request actually does. So this piece starts there, then comes back to what Gimlet is selling and where the skepticism belongs.
Why inference is splitting across chips
Generating an answer from a large model is not one uniform task. It has two phases with opposite appetites.
The first phase is prefill. The model reads the entire prompt at once and builds up its internal context. This is heavy matrix multiplication, and it rewards raw compute density, which is what a modern GPU is built to deliver.
The second phase is decode. Having read the prompt, the model produces the answer one token at a time, and each new token has to look back over everything generated so far. This phase is not limited by how many multiplications a chip can do. It is limited by how fast the chip can move data in and out of memory. Here, a GPU's enormous compute is partly wasted, and a chip designed around fast on-chip memory can do better.
One inference request, two very different workloads
The two phases of generating an answer stress hardware in opposite ways. That gap is what multi-silicon inference exploits.
Reads the whole prompt at once and builds the context.
Rewards raw matrix-multiply throughput: dense GPUs.
Generates the answer one token at a time.
Rewards fast memory access: SRAM-based accelerators.
Educational summary of prefill and decode inference phases. See article for context.
Run both phases on the same chip and one of them is always leaving the hardware underused. The idea behind disaggregated inference is to stop doing that: run prefill where compute is cheap, run decode where memory bandwidth is cheap, and hand the coordination to something in between. Agentic workloads, which fire many short model calls in sequence rather than one long chat, make the imbalance sharper, because they spend proportionally more time in the phases where a single chip type sits idle.
What Gimlet is actually selling
Gimlet describes a platform that disaggregates a model and places each phase on the most appropriate silicon, integrating GPUs, purpose-built AI accelerators with SRAM-based architectures, and CPUs into a single system with, in the company's words, intelligent workload orchestration. The pitch is not a faster chip. It is a scheduler that treats a fleet of different chips as one pool and routes each piece of work to the hardware that runs it best.
NVIDIA
AMD
MicrosoftThe company reports headline gains of up to 10 times in throughput and interactivity, along with lower latency and a smaller infrastructure footprint. Those are vendor figures, and they should be read as best-case rather than typical, but the direction is consistent with the physics: if each phase runs on hardware suited to it, utilization goes up and cost per token comes down. Gimlet also says it tripled its customer base by early 2026 and has signed one of the top three frontier labs and one of the top three hyperscalers, with billions of dollars in contracted revenue and infrastructure scaling to hundreds of megawatts. Founder and chief executive Zain Asgar frames the edge as software: the company can deliver performance, he says, because it slices and orchestrates workloads across different types of hardware.
The pitch is not a faster chip. It is a scheduler that treats a fleet of different chips as one pool.
The core of the multi-silicon thesis
The orchestration layer is the actual product
Slicing a model across chip types is only useful if something schedules the pieces well. That scheduler is the hard part.
Source: Gimlet Labs Series B announcement, September 4, 2026. Simplified for illustration.
The hard part is not the chips
Mixing silicon is easy to describe and hard to do well, and the difficulty is exactly where Gimlet is trying to build a moat. Splitting a model across chip types introduces new costs: the intermediate state produced by prefill has to move to wherever decode runs, and that movement eats into the efficiency the split was supposed to create. Get the orchestration wrong and a multi-silicon system is slower and more expensive than just buying more GPUs. Get it right and the gains are real. The entire value sits in scheduling, data movement and knowing which workload belongs where, not in owning any particular chip.
This is also why the competitive picture is unsettled. Cloud providers already build their own inference stacks, chipmakers have every reason to keep customers on a single architecture, and a neutral orchestration layer sitting above all of them is a useful position and a contested one at the same time. The investor list reflects that ambiguity: a chip designer and a hyperscaler's venture arm both backed a company whose product is, in part, about not being locked to any one of them.
The pattern underneath the deal
Strip away the specifics and Gimlet is an argument for heterogeneity over standardization at the infrastructure layer. The bet is that the winning approach is not one chip, one vendor or one architecture, but a system smart enough to route each piece of work to whatever runs it best, and to keep doing so as the hardware underneath keeps changing. That is a bet against lock-in, made at the silicon layer.
The same logic shows up one level higher, at the model layer, and it is worth naming because it is the same idea wearing different clothes. Just as no single chip is best for every phase of inference, no single model is best for every task an application handles, and committing an entire product to one of either is a bet that today's best choice stays best. Platforms built to route across models rather than hard-wire one, the way Metir AI treats the underlying model as a swappable choice rather than a fixed dependency, apply at the application layer the same principle Gimlet is applying at the hardware layer: keep the expensive decision reversible, because the ground keeps moving.
What to watch next
Three things will tell you whether the multi-silicon thesis holds. The first is independent benchmarks: the up-to-10x figure is the company's, and third-party numbers on real agentic workloads will separate marketing from mechanism. The second is whether the top-tier lab and hyperscaler Gimlet has signed expand their usage or stay at pilot scale, since orchestration software lives or dies on production trust. The third is how incumbents respond, because if disaggregated inference works, both chipmakers and clouds have strong incentives to build or buy their own version rather than route revenue through a neutral layer. The physics behind Gimlet's pitch is sound. Whether a standalone company can own the orchestration layer, rather than watch it get absorbed by the platforms it sits on, is the open question the $300 million is betting on.
Sources:
- Now Valued at $3 Billion, Gimlet Labs Raises $300 Million in Series B (GlobeNewswire, Sept 4, 2026)
- Gimlet Labs nabs $300M for its disaggregated inference platform (SiliconANGLE, Sept 4, 2026)
- Andreessen-Backed AI Startup Gimlet Is Valued at $3 Billion in New Round (Bloomberg, Sept 4, 2026)
- Gimlet Labs Raises $300M at $3B With Arm and Microsoft Backing a Chip-Agnostic Inference Layer (FourWeekMBA, Sept 2026)
- Gimlet Raises $300M to Mix AI Chips (Quasa, Sept 2026)
Image credits
Header and in-body photograph: NVIDIA H100 data-center GPUs by Geekerwan, via Wikimedia Commons, licensed under CC BY 3.0.