Agent Engineering
From Retrieval to RAG
Search returns possibly relevant chunks. RAG must turn them into bounded, citable, and measurable evidence.
Where this lesson sits: The entry point for vector retrieval is Long-Term Memory: Vector Retrieval, earlier in this topic; the product view of RAG is in Chapter 1, in RAG — Retrieval-Augmented Generation and The Cost and Optimization of RAG. This lesson skips those fundamentals and only covers putting Milvus retrieval into a reliable RAG pipeline; wrapping retrieval as an Agent tool comes in Chapter 4, in Milvus as an Agent Knowledge Tool.
Online path
1
Embed query
Use the same model as ingestion.
2
Retrieve
Tenant/ACL filter plus Top-K search.
3
Rerank
Rerank, deduplicate, and budget tokens.
4
Generate
Answer only from evidence and cite sources.
Offline ingestion sets the ceiling
Chunking
Split on titles, paragraphs, and semantic boundaries with modest overlap. Large chunks add noise; tiny chunks lose context.
Metadata
Keep source ID, version, section, tenant, ACL, and update time for citations, filters, and deletion.
Versioning
Re-embed when content or the model changes. Blue/green collections avoid mixing incompatible vectors.
Retrieval toolbox
| Method | Purpose | Caution |
|---|---|---|
| Scalar filter | Tenant, ACL, time, language | Enforce access during retrieval, never only in the prompt |
| Vector + keyword hybrid | Semantics plus exact SKUs/error codes | Raw score scales are not directly comparable |
| RRF | Fuse several rankings | Simple and stable; rerank the resulting candidates |
| Partition / TTL | Hot-data isolation or expiration | Advanced options; start with clean collections and filters |
Evaluate two layers: retrieval Recall@K, MRR/nDCG, and filter correctness first; then answer faithfulness, citation accuracy, and appropriate refusal. A fluent answer can hide failed retrieval.
Takeaway RAG is more than stuffing search hits into a prompt. Chunking, metadata, access filters, fusion, and citations make retrieval-augmented answers reliable.