Programming Fundamentals · Data Structures Inside the LLM

Vectors: RAG Retrieval Is Finding Nearest Neighbors

When we learned RAG, we said Embedding turns a passage into a string of numbers. What are those numbers, really? The answer is almost too simple—coordinates. Each sentence gets a spot on a “semantic map”: closer in meaning means closer in place. So “retrieving documents” becomes a game every kid knows: find the neighbors closest to you on the map.

Play the map first · nearby meaning = nearby coordinates

Below is a mini semantic map: 12 words already seated by Embedding, naturally clustering into three “districts.” Drag the black ❓ query point (or click anywhere on the map to place it). Watch for: the lines always point to the 3 nearest words, distances update live, and the closest one wears 👑. Drag ❓ between two districts and see the neighbors swap.

Try dragging ❓ into the “Food district,” then over to the “Tech district.”
That’s the entire principle of RAG retrieval. Every passage in your knowledge base gets a coordinate from Embedding (in the real world it’s 1536 dimensions, not 2—but the idea is the same); when you ask, the question gets a coordinate too; then you find the passages closest to the question and stuff them into context for the LLM. The secret behind “semantic search is smarter than keyword search” is that “milk tea” and “mango pomelo sago” share zero characters, yet sit right next to each other on the map.
Then speed · brute-force vs building highways

The dumbest way to find nearest neighbors is to measure distance to every point. Sixty points are fine—but RAG knowledge bases often hit millions of chunks, and recommenders hit billions. Below, the same map scatters 60 points; 🌟 is your query. Click “Brute-force” first and count the steps, then “HNSW layered hops” to compare—watch how the blue jump lines “big hop first, then small hops.”

🐢 Brute-force (full scan)
—
distance calculations
🚀 HNSW layered hops
—
distance calculations
HNSW intuition: build the highways first. Besides the full map at the bottom, store a few increasingly sparse “shortcut graphs”—queries start from the sparsest top layer, lock onto a region in a few big hops, then descend layer by layer with finer jumps. Another case of trading space for time.
With 60 points it’s 60 vs 8; at hundreds of millions, that’s “wait a few minutes” vs “milliseconds.” Brute-force cost grows one-for-one with data size; HNSW chops huge search regions with a few hops per layer, so even hundred-million-scale vectors need only dozens of steps. What vector databases (Milvus, Pinecone, FAISS—names you’ll meet sooner or later) really sell is building and maintaining those “shortcut graphs.”
What does this have to do with AI?
📚

RAG retrieval

Questions and documents both become coordinates; find the nearest passages and feed them to the LLM—the “answer from a knowledge base” you use every day is exactly these two animations under the hood.

🖼

Image search

Images can be Embedded into coordinates too. Snap a photo of a sofa to find matches—that’s finding your photo’s neighbors on a semantic map of hundreds of millions of images.

🎯

You might also like

Your taste is a coordinate; every song and show is one too. What recommenders do all day: find the content closest to you and serve it up.

✅ What this lesson wants to share