The reports started appearing in developer forums last week. Users of OpenAI's Codex โ the coding agent embedded in ChatGPT โ were watching their quotas evaporate at rates that made no sense relative to their actual usage. Not edge cases. Not power users running automated pipelines. Ordinary developers hitting their monthly limits in days, sometimes hours.
Tibo, an OpenAI developer advocate, confirmed the issue. Three problems were identified: inefficient image context compression, runaway resource consumption from the Computer History feature, and background title generation triggering on every message. Quotas were reset for affected users.
The apology is fine. The quota reset is fine. But the technical admission buried in that response should worry anyone building on multimodal AI infrastructure. Because the real story isn't about quota math. It's about what happens when a company ships multimodal features faster than its inference infrastructure can handle them.
The Mechanics of the Drain
Let's break down what's actually happening under the hood.
Codex pricing operates on a composite calculation: request count plus context length. When you send a conversation to the model, every token in that conversation โ including tokens derived from images โ gets processed during the prefill phase. This is where the cost structure breaks down.
Here's the problem with image tokens specifically. OpenAI's vision pipeline uses a CLIP ViT-L/14 encoder that produces roughly 256 patch tokens per image. Those tokens enter the context window alongside text tokens. But they don't behave like text tokens during compression.
Standard token-level compression strategies โ importance-based pruning, for instance โ work reasonably well for text because semantic information is distributed relatively evenly across tokens. Visual tokens are different. They contain both spatial redundancy and semantic redundancy simultaneously. Compressing them effectively while preserving critical information is a fundamentally harder problem.
When a conversation contains many images and undergoes multiple compression cycles, each compression pass introduces additional resource waste. The compressed output doesn't retain the information density of the original, so subsequent passes require more tokens to represent the same content. The system degrades multiplicatively, not linearly.
The Computer History feature makes this worse by an order of magnitude. For Mac users who enable it, Codex receives a continuous stream of screen captures โ not static images but a de facto video feed. This fundamentally changes the temporal dimension of the context. The context compression mechanisms were designed for static multi-image inputs, not high-frequency visual streams. Every compression cycle on that stream carries a marginal cost significantly higher than the design specification anticipated.
And the title generation? A seemingly trivial feature. But if it triggers on every message interaction rather than once at conversation initiation, it represents a hidden model call that users never see and never consented to in their mental cost model.
The Cache Hit Rate Signal
Here's what I find most interesting from a systems perspective.
Tibo acknowledged that some users experienced cache hit rate degradation. This is a quiet admission with loud implications.
Prefix caching works by storing the KV cache of the initial token sequence in a conversation. When subsequent requests share that prefix, the system reuses the cached computation instead of recomputing it. This is how OpenAI keeps inference costs manageable for long conversations.
But compression changes the token sequence. When a conversation gets compressed, the resulting token sequence no longer matches the original sequence stored in the cache. The prefix cache invalidates. The system must recompute the entire KV cache from scratch.
Now think about what this means in practice. A conversation that gets compressed multiple times โ say, a long debugging session with screenshots โ experiences repeated cache invalidation. Each invalidation forces a full recomputation. The cost isn't just the compression overhead; it's the lost caching benefit across the entire context window.
The fact that this wasn't caught before shipping suggests the monitoring systems weren't tracking cache hit rates as a first-class metric for multimodal conversations. Or they were, and the degradation was gradual enough to escape threshold alerts. Either way, it points to a monitoring blind spot in OpenAI's infrastructure.
The Cost Invisibility Problem
Stepping back, this event exposes a structural issue with usage-based pricing in multimodal AI products.
Users have a mental model of what a "request" costs. Text in, text out. But multimodal inputs break that model. A single screenshot can consume more tokens than an entire text conversation. A screen capture stream at even modest frequency can consume more tokens than a novel.
The asymmetry between user expectation and actual cost is the real systemic risk here. Not just for OpenAI, but for every AI product that accepts multimodal input and charges based on token consumption.
The industry response will likely be one of two paths. Either products move toward more transparent per-token billing with real-time usage dashboards, or they bundle multimodal features into fixed-price tiers with aggressive compression to keep costs manageable. The former is better for users but exposes the true cost structure. The latter maintains the illusion but risks recurring incidents like this one.
The Architectural Lesson
From an infrastructure perspective, the deeper lesson is about how quickly multimodal capabilities outpace optimization.
OpenAI's inference stack was optimized for text-dominant workloads. The prefill optimization, the cache strategies, the compression algorithms โ all designed around the statistical properties of text tokens. Visual tokens behave differently, and the system's assumptions break down.
The fix isn't a patch. It's architectural. More efficient visual tokenizers, potentially with larger patch sizes. Cache strategies robust to compressed token sequences. Possibly even hardware-assisted compression using the NPUs already sitting in Apple Silicon and other client devices.
The companies that solve multimodal inference efficiency will have a structural cost advantage that pricing alone cannot match. This event is an early signal that the current generation of infrastructure is not there yet.
The quota resets will smooth things over. The patch will ship. But the underlying cost structure remains โ and it will keep surfacing in unexpected places until the architecture itself evolves to match the multimodal reality of modern AI usage.
