Cloudflare disclosed on October 26, 2026 that a configuration mismatch in its Workers AI service, combined with a bug in the SGLang inference engine, caused some long-prompt requests to the GLM 5.3 model to receive another customer's data. The company identified 155 affected responses between August 27 and September 9 and found no evidence of deliberate exploitation. All details below come from Cloudflare's remediation post.
What happened
An inference server keeps a key-value (KV) cache, the stored attention state for tokens already processed, so it does not recompute them. In a disaggregated setup, one set of nodes does the prefill (reading the prompt) and another does the decode (generating the answer), and the cache is transferred between them.
Cloudflare launched GLM 5.3 on Workers AI on August 27 with a prefill KV pool of 681,280 tokens, smaller than the model's 1,048,576-token context window. The company says it knew of the mismatch before launch but did not treat it as a problem, because its long-context tests returned successful HTTP responses.
Cache pool versus context window
Tokens. A prompt longer than the pool could not be held safely, but the context window allowed it.
The launch pool held about 65% of the context window (681,280 / 1,048,576).
The second ingredient: an engine bug
The mismatch alone should have produced rejections. Cloudflare says a bug in SGLang's disaggregated mode turned a rejection into a leak: an over-length request was marked rejected and replaced with a one-token placeholder instead of being aborted. The admission queues checked whether a request was finished, not whether it was rejected, so the request continued through KV transfer. The decode stage then attended over stale KV pages left by earlier requests, and the client received an HTTP 200 containing another user's data.
Affected prompts were all longer than 681,280 tokens, and Cloudflare said the affected requests looked like ordinary long-context use.
We only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.
Cloudflare, on the testing gap
Timeline
| Date (UTC) | Event |
|---|---|
| Aug 27 | GLM 5.3 launches with the smaller pool |
| Sep 8, 23:11 | Researcher report received via HackerOne |
| Sep 9, 13:22 | Incident declared and leak reproduced with two accounts |
| Sep 9, 14:00 | Model set to private, returning HTTP 400 |
| Sep 9, 14:52 | Prefill nodes rolled to corrected configuration |
| Sep 9, 19:40 | Model restored to public |
| Oct 2 and Oct 19 | SGLang v0.5.21 and v0.5.22 released |
| Oct 26 | Coordinated publication |
Source: Cloudflare. From report to private took under 15 hours; the public disclosure came about seven weeks after the fix.
Remediation
Cloudflare matched the GLM 5.3 configuration to GLM 5.2, raising the prefill pool to about 2.04 million tokens, roughly twice the context length. It patched SGLang so requests with pending rejections are retired at the admission queue with their original error before any allocation or transfer, and validated both changes against the researcher's reproduction and internal canaries. Upstream fixes landed in SGLang v0.5.20, v0.5.21 and v0.5.22, with deferred KV release enabled by default, bounded transfer batches, and per-transfer checksums added for observability. Deployment checks now compare KV pool size against context length.
Longer term, the company says it is developing page-level integrity tags in SGLang to detect reads of pages a request does not own, noting that an internal prototype cost under 1% in throughput and tail latency but was disabled for reliability reasons, and applying formal verification to its RDMA-based KV transfer protocol.
What the incident teaches
- A 200 is not correctness. Success codes and throughput checks cannot detect a response that is well formed but wrong. Cloudflare names this as its own gap.
- Two safe-looking components can combine unsafely. A known configuration mismatch and a rejection-path bug were each tolerable alone.
- Shared inference needs isolation checks. Multi-tenant caches are a new class of boundary, similar in spirit to the cross-tenant risks discussed in our coverage of AI provider concentration.
- Scope is narrow but real. 155 responses is small against total traffic, but the content of each is by definition someone else's data.
For teams choosing where to run models, the practical question is what tests a provider runs on output integrity, not only availability. Metir supports several model providers, so an incident at one host does not have to halt a workflow.
Sources:
- KV cache exposure remediation - Cloudflare Blog
- Everything That Happened in AI Today, Wednesday, October 28 - The Neuron
Image credits
Header and in-body image: Cloudflare office entrance area by HaeB, Wikimedia Commons, CC BY-SA 4.0 (source).