OpenAI put unusually specific numbers on ChatGPT's storage layer on September 11, 2026. Habitat, its internal online-data platform, now supports products used by more than one billion people each week, serves over 70 million requests per second across almost 40 regions, and sits in front of more than 500 petabytes of data. The architecture began at DevDay 2023 as a Python library connected to Azure Cosmos DB.
OpenAIOpenAI
Microsoft
AzureHabitat is a control layer, not one database
Product teams use Habitat without deciding where each request should go. The service handles schema lookup, routing, authorization, encryption, serialization, rate limiting, connection pooling, caching, data residency, and multi-tenant isolation. Underneath it sit Cosmos DB, caches, blob storage, and change-data-capture pipelines feeding systems such as Kafka, Databricks, and Rockset.
That design intentionally limits what application developers can ask the hot path to do. Habitat exposes object-and-edge operations rather than arbitrary SQL joins or graph traversals. More complex analytical reads move to isolated secondary systems. The tradeoff is extra friction for unusual queries in exchange for protecting the shared online service from one expensive query taking down unrelated products.
Habitat's September 2026 scale
A snapshot of OpenAI's disclosed online-storage workload. The units describe different dimensions and should not be compared as one series.
Source: OpenAI engineering, September 11, 2026. Figures are company-reported and rounded.
Why OpenAI waited to replace Python
OpenAI says Habitat's Python implementation once served more than 20 million requests per second. Rather than rewrite it immediately, the team spent a year addressing more urgent scale and reliability problems. In the second quarter of 2026, two engineers using Codex and GPT-5.5 rewrote the service in Rust.
The Rust version now handles 95 percent of production requests. OpenAI reports six times better CPU efficiency and 15 times better memory efficiency, plus lower average and tail latency. These are internal measurements, not an independent benchmark, and the company has not published absolute hardware cost or latency figures.
The deeper lesson is sequencing
The fashionable reading is that Rust replaced Python because Rust is faster. The more useful lesson is that OpenAI did not begin with the final architecture. It used a lightweight client library, converted that library into a service as coordination costs rose, constrained the query model to protect shared capacity, and only then paid for a language rewrite when resource efficiency became the bottleneck.
The engineering story is less 'Rust beats Python' than 'optimize the bottleneck you actually have, in the order it becomes expensive.'
Metir analysis
That sequence matters for smaller AI products. Most do not need a 70-million-request platform, but they do need explicit authorization, bounded queries, tenant isolation, and a path to move analytics away from transactional workloads. OpenAI's disclosure also makes the physical scale behind apparently simple chat interactions easier to see. Model inference is only one part of the system; identity, conversation history, settings, tools, and agent state all depend on conventional distributed storage working reliably.
OpenAI says a second installment will explain multi-tenant reliability and how its storage partnership with Azure Cosmos DB supports the cited scale. Until then, several important details remain undisclosed, including the breakdown of the 500 petabytes, failure rates, regional consistency choices, and the full cost of operating the platform.
Sources:
- Rapidly scaling online storage to serve over 1 billion ChatGPT users | OpenAI
- TAO: Facebook's distributed data store for the social graph | USENIX
- Azure Cosmos DB documentation | Microsoft
Image credits
The Pioneer Building, San Francisco, HaeB, via Wikimedia Commons, licensed under CC BY-SA 4.0.