Programming Fundamentals · Linear Structures: You Use Them Every Day

Queues: An Agent's Work Gets Done in Line

Last lesson's stack was “last in, first out.” This one flips the direction: first in, first out—in one end, out the other, like the lunch line at a cafeteria. Don't dismiss it for looking plain: AI services that survive ten thousand questions at once, and Agents that finish a string of tasks in order, both rely on this line. Today you're the dispatcher: start a pipeline yourself, push it into backlog, then save it.

Hands-on · you're the dispatcher

Users on the left keep sending requests into the middle queue; the Agent worker on the right pulls them from the exit end in order (first come, first served). The pipeline starts itself when it scrolls into view. How to play: crank “Request rate” all the way up and watch how fast the queue turns red; then raise “Processing speed” and see the backlog get digested.

👥
User requests
Producer
(queue is empty)
😴
Agent worker
Consumer
0
In queue
Pipeline not started yet
Enqueued 0 ｜ Done 0
2.0/s
2.0/s
Gray = idle ｜ Green = healthy ｜ Red = backlog over 8
That backlog you just saw is peak shaving in action. Requests spike (the peak); workers can't keep up—that's fine: the queue catches them first, nothing dropped, no cutting in line. When demand dips (the valley), workers slowly clear the line. Without this queue, anything beyond capacity gets rejected on the spot. When AI feels slow at peak hours but rarely hard-errors, the queue behind it is lining you up. The flip side: a chronically empty queue means you overstaffed workers—queue length is the most honest dashboard of system health.
30 seconds · stack vs queue: the only difference is which end you take from

Put A, B, and C in, then take them all out—one-click play. Watch the “order they come out” on both sides.

🥞 Stack (last lesson's friend)

Same end in, same end out

Order out: —

🚶 Queue (today's star)

In one end, out the other

Order out: —
Put A, B, C in order, then take them all out
C → B → A vs A → B → C. The only difference between the two structures: which end you take from. A stack takes from the same end—perfect for “backing out the way you came” (undo, function return). A queue takes from the other end—perfect for “first come, first served.” Ways of organizing aren't ranked higher or lower—only fit or unfit.
Its real form in the AI world
📋

An Agent's todo list

After an Agent breaks work into subtasks, it pushes them into a queue and works them in order: research → draft → self-check. Planned first, executed first—no skipping, no dropping. That order isn't intelligence; it's a queue.

🚦

API rate-limit queue

Model APIs only accept a fixed number of calls per minute. Extra requests aren't thrown away—they line up for the next window. When your program returns a beat late, it's often waiting in that queue.

📮

Message queue

In big systems, services don't shout at each other—they write work as messages into a queue, and the other side pulls at its own pace. That's the industrial version of the pipeline you just ran; the jargon is message queue (Kafka and RabbitMQ are both that).

What this lesson wants to share