Token Cost Engineering · 7 / 13

Input-Dominated: a 62:1 I/O Ratio

A plain chatbot is one question, one answer; an Agent is a “think → act → observe → think again” loop. The key idea: every turn's Input includes the full history—more turns means longer Input and ballooning cost.

ReAct loopinput-dominatedI/O RatioCumulative bloat

Chatbot vs Agent: totally different billing structures

A plain chatbot asks “tell me a joke” (10 Tokens) and answers with 150 Tokens—I/O Ratio ≈ 1:15, output-dominated. An Agent doing a code-fix task: System Prompt, tool returns, and past outputs all get re-read every turn. After three turns, total Input 16,790 and total Output 270—I/O Ratio 62:1, input-dominated.

To get a 270-character result, you paid a 16,790-character “reading fee”. That's the Agent's input-dominated nature.

Plain chatbot Q&A vs Agent loop execution

A chatbot ends after one turn; every Agent turn enters with the full history, so Input accumulates. (Figure: from the author's internal share deck)

I/O Ratio: the Agent's input-dominated nature

Three-turn breakdown of a code-fix task: total Input 16,790, total Output 270, I/O Ratio 62:1. (Figure: from the author's internal share deck)

Interactive Demo · The real bill for a “simple” task

Task: “Analyze this Excel sheet, find the top-selling product, then generate a chart.” Hit “Next turn” and watch cumulative Input snowball each round.

Excel analysis task · 5 turns

Turn-by-turn breakdown and real bill for the Excel task

Five turns: cumulative Input 31,460, Output 450, about ¥0.014–0.026 total. Note: this is one successful run—hidden costs still include failed retries, debugging, and iteration. (Figure: from the author's internal share deck)

The math of bloat: why it's quadratic

Suppose a task takes N turns, System Prompt length S, and each turn adds about Δ (output + tool return). Turn N's Input ≈ S + Q + Δ×(N-1), and total Input is the sum across all turns—hiding the arithmetic series 1+2+3+…+(N-1). At +500 Tokens per turn, five loops already push total Input past 15,000; double the turns and cost nearly quadruples.

Agent framework (SWE-bench measured)	Avg. I/O Ratio	Notes
Simple RAG Agent	10:1 ~ 20:1	Retrieve + answer
OpenHands	20:1 ~ 50:1	Code-fix tasks
AutoGPT-class	30:1 ~ 100:1	Open-ended tasks, many loops

That's why KV Cache (lesson 11) is decisive for Agents: if every turn re-reads the same prefix, a cache hit is a 5× price gap. It also explains why you should watch I/O Ratio—above 50:1 usually means the Agent is spinning / idle looping, so optimize the workflow or degrade the task.

Key Takeaways

✓

Agents are input-dominated: every turn's Input carries the full history; total volume grows roughly with the square of turn count.

✓

One run looks cheap; scale is where it bites: a ¥0.02 task × failed retries × a million calls is the bill-shock moment.

✓

Treat I/O Ratio as a health metric: >50:1 means the Agent is spinning—fix the workflow before you chase savings.

Source: Adapted from the author's internal team share “AI Token Cost Engineering Strategies,” section “Billing Mechanics for Agentic Apps.” Industry data references SWE-bench studies of Agent Token consumption (see Finale reading list). For Agent loop fundamentals see Hands-On Practice.