Token Cost Engineering · 8 / 13

Four Agent Cost Traps and the Circuit Breaker

Last lesson was still “one successful run.” In the real world, Agent bill accidents come from four directions: tool-return explosion, thinking tax, infinite loops, and history snowball. Each has an engineering fix.

Tool truncationthinking taxLoop circuit breakerHistory compression

Interactive Demo · Tear down the four traps one by one

Trap 1 · Tool-return information explosion

User says “pull every user's orders from the database,” the Agent's SQL tool returns 10,000 rows ≈ 500,000 Tokens. Those 500k Tokens get stuffed into the next turn's Input: you trip the expensive tier, maybe blow the context window, and the model “gets lost” under overload so output quality drops. The fix is a truncation guard on every tool:

def safe_tool_call(tool_func, *args, max_tokens=2000, **kwargs): result = tool_func(*args, **kwargs) result_str = json.dumps(result, ensure_ascii=False) estimated = len(result_str) * 0.5 # 粗略估算 Token if estimated > max_tokens: # 保留前后各一段 + 中间标记，提示模型缩小范围 truncated = result_str[:1000] + "\n...[truncated]...\n" + result_str[-500:] return { "status": "truncated", "preview": truncated, "total_records": len(result), "message": f"Result too long (~{int(estimated)} Tokens), truncated. " "Narrow the query if you need the full data." } return result

Tool-return information explosion and truncation strategy

10,000 SQL rows ≈ 500k Tokens: truncation guards are table stakes for the Agent tool layer. (Figure: from the author's internal share deck)

Trap 2 · The invisible bill of thinking Tokens

Qwen-Plus thinking mode, DeepSeek-R1, o1 and similar models emit a “thinking process”: users may never see it, but it's all billed as Output—and the unit price is 4× (Qwen-Plus non-thinking output 2 ¥/M, thinking mode 8 ¥/M). Same Agent task with thinking on: visible output unchanged (450 Tokens), thinking process +2,000 Tokens, output spend jumps +2,078%.

Task type	Thinking mode	Why
Simple retrieval	❌ Off	No deep reasoning needed
Data cleaning	❌ Off	Rules are clear—no “thinking” required
Complex reasoning	✅ On	Worth paying for accuracy
Code generation	⚠️ It depends	Off for simple functions; on for complex architecture

Advanced fix: use a ~0.6B tiny model as a front-door triage—spend a few 厘 first to decide whether this request needs deep thinking, then route to the right mode. That's the concrete shape of lesson 3's “T2 backs up T0.”

The model's “inner monologue” is burning money at 4× the unit price: tier thinking mode by task. (Figure: from the author's internal share deck)

Trap 3 · Infinite loops

Agent fixing a bug: fix A → error B → fix B → error A (back to square one) → … still spinning at turn 15. If Input grows 1,000 Tokens per turn, 20 turns cost 13×; worse, the user waited 5 minutes with nothing done. The fix is a forced circuit breaker—graceful exit when any of three conditions hits:

class AgentExecutor: def __init__(self, max_rounds=10, max_tokens=50000): ... def execute(self, task): while not task.is_complete(): self.round_count += 1 # 熔断 1：轮次上限 if self.round_count > self.max_rounds: return self._graceful_exit("Hit max execution turns") # 熔断 2：Token 预算 if self.total_input_tokens > self.max_tokens: return self._graceful_exit("Hit Token budget cap") # 熔断 3：死循环检测（连续 3 轮输出相似度 > 90%） if self._detect_loop(): return self._graceful_exit("Possible infinite loop detected") result = self._run_one_round(task) self.total_input_tokens += result.input_tokens

On graceful exit, return rounds_executed, tokens_consumed, and partial_result—a half-finished artifact beats a black hole.

Agent infinite loops and three circuit-breaker strategies

Turn cap, Token budget, loop detection: three fuses so a task never waits forever. (Figure: from the author's internal share deck)

Trap 4 · History snowball

The standard (wrong) approach stuffs the full history into Input every turn. The better approach is fixed prefix + compressed history + last N turns: never compress the System Prompt (preserve the cache prefix), keep the last 3 turns verbatim, and squash older history into one summary sentence with a small model.

Approach	Turn-10 Input	Notes
Unbounded growth	~50,000 Tokens	Includes full history
Sliding window (last 5 turns)	~12,000 Tokens	Loses early context
Fixed + summary + last 3 turns	~6,000 Tokens	Keeps what matters, controls length

Combined checklist and three red lines

Control point	Strategy	Expected gain
Tool returns	Truncate + summarize, cap 2k Tokens	Stop single-turn explosions
History management	Fixed prefix + compress old history	Cut Input 50%+
Loop control	Circuit-breaker mechanism (turns / Tokens / loop detect)	Stop bottomless pits
Thinking mode	Enable by task tier	Cut Output cost ~4×
Model selection	Small models for simple subtasks	Lower unit price
Cache use	Fixed System Prompt, hit KV Cache	Cut Input cost ~90%

Red line	Suggested threshold	Consequence	Response
Per-turn Input	< 32k Tokens	Jump into expensive tier	History compression + tool truncation
Total turns	< 10 turns	Cost grows exponentially	Circuit-breaker mechanism
I/O Ratio	Watch > 50:1	Agent is “spinning”	Optimize workflow or degrade the task

Key Takeaways

✓

Cap tool returns at 2k: truncate + summarize + tell the model to narrow scope—stop single-turn Input explosions.

✓

Tier thinking mode by task: the invisible inner monologue still bills as Output, at 4× the unit price.

✓

Circuit breakers are the Agent's fuse: hit any of turns, Token budget, or loop detection → graceful exit.

✓

Manage history with “fixed + summary + last 3 turns,” half the cost of a blunt sliding window without amnesia.

Source: Adapted from the author's internal team share “AI Token Cost Engineering Strategies,” section “Billing Mechanics for Agentic Apps.” Product angles on Agent freezes and fool-proofing are covered in Hands-On Practice; for context compression also see Harness Core · Context Overflow.