GLM's Short-Output Game: the 200-Token Cliff
Look at GLM-4.6's pricing structure and you'll spot a clever design: it doesn't tier by input length—it tiers by output length, with the boundary at 200 Tokens. Output 199 vs output 201: completely different prices.
| Metric | Output ≤ 200 | Output > 200 | Change |
|---|---|---|---|
| Output unit price | 4 ¥/M | 7 ¥/M | +75% |
| Input unit price | 1 ¥/M | 1.5 ¥/M | +50% |
The easy-to-miss part: once output crosses 200 Tokens, the thousands of input Tokens you already sent also rebill at the higher price. Two extra words of output, and the whole request gets a retroactive price bump.
This mirrors the marginal-cost curve of inference compute. Short output (<200 Tokens) is light: Decode pressure is low, KV Cache footprint is limited, and it may finish in tens of milliseconds. Once output gets long, every extra Token grows KV Cache VRAM and adds another Attention round—cost scales nonlinearly.
Scenario: extract structured JSON from user reviews (sentiment / aspects / pain_points / suggestions). The Prompt is already clean, but you can't predict how much each review will yield—a simple one is 150 Tokens; two extra complaints push it to 230. Drag the slider to feel this “structural conflict.”
| Strategy | What you do | Trade-off |
|---|---|---|
| Task splitting | Split extraction into multiple calls; pull only 1–2 fields each time | More calls, higher latency |
| Field tiering | Extract core fields in real time; fill secondary fields async or in post-processing | More architectural complexity |
| Accept volatility + monitor | Allow occasional tier jumps, but monitor the overall distribution | Cost is controllable, not optimal |
| Model degrade | Route price-sensitive high-volume tasks to high-volume models like Qwen-Flash | May trade a bit of accuracy |
The key question: where does this task's output naturally land? If most requests sit at 100–150 and only rarely cross 200, you can live with it. If the median is already 180–220, the task sits on the cliff—you must redesign task granularity, or switch to a model that doesn't tier by output. The same pattern shows up in codegen: a 20-line function easily burns 100+ Tokens, and a slightly more complex edit tip crosses 200.
GLM-4.6 tiers by output length; 200 is the cliff: output +75%, input retroactive +50%.
Pricing mirrors compute cost: long-output KV Cache and Attention grow nonlinearly—vendors use price to push you toward short, sharp work.
Check the output distribution first, then pick a strategy. Tasks whose median sits on the cliff need a granularity cut—or a different billing model.
Source: Adapted from the author's internal team share “AI Token Cost Engineering Strategies,” section “GLM-4.6's Short-Output Game.” Prices are the author's discounted rates at the time; check Zhipu's open platform for live quotes.