SGLang incident puts request isolation under scrutiny

LMSYS described a request-isolation incident involving SGLang in Cloudflare's GLM 5.3 deployment. The report identified an exposure window from August 27 to September 9 and said other Workers AI models were not affected by that incident.

The failure involved a disaggregated serving path, where prompt processing and answer generation occur in separate components. A rejected request could still proceed through a handoff, while asynchronous transfers from an aborted request could reach memory that had been reused. The published fixes enforce rejection and defer memory release until outstanding transfers are resolved.

Isolation lives below the prompt

The incident concerns the machinery that serves model requests. An application can send an entirely ordinary prompt and still depend on the serving system to keep request state separate. A well-behaved model cannot compensate for infrastructure that attaches the wrong cached information to its execution.

This distinction matters for buyers comparing hosted and self-managed inference. Security review should include the serving stack, its version, and the provider's response process. Looking only at model weights or content safeguards leaves a separate class of failures outside the review.

A useful test plan should concentrate on transitions. What happens when a request is rejected, cancelled, interrupted, or moved between workers? What happens when memory is reused under pressure? These are less visible than normal successful generation, yet they are precisely where ownership and lifetime assumptions can break down.

Exercise failure paths deliberately

Teams can use synthetic requests with distinct, non-sensitive markers to check isolation during controlled stress tests. The objective is to detect any unexpected mixing of state, while avoiding real customer data. Such tests should run alongside ordinary performance measurements rather than being postponed until after a speed optimization ships.

Capacity settings also need to agree across components. A front-end limit, a scheduler's assumption, and a worker's actual memory budget should not describe three different operating envelopes. When they disagree, rejection handling becomes part of the normal execution path and deserves corresponding attention.

For operators affected by a serving vulnerability, patch status and incident scope should be communicated separately. A fix explains how recurrence is addressed; it does not, by itself, establish which historical requests were exposed.

The disclosure is a reminder that dependable inference includes correct handling of unfinished work. Cancellation and cleanup should preserve request boundaries just as rigorously as the path that returns a successful answer.