The Art of Tool Design

Milvus as an Agent Knowledge Tool

Let the Agent decide when company knowledge is needed. The tool retrieves evidence; the Agent turns the ToolMessage into an answer.

One tool call
1

Agent decides

The question needs internal knowledge, so it selects search_knowledge.

2

Embed + Top-K

The tool embeds the query and searches with an ACL filter.

3

ToolMessage

Return chunks, sources, and scores—not a fabricated answer.

4

Agent answers

Cite evidence and say when it is insufficient.

Put boundaries in the tool description
The client and encoder below reuse the connection and embedding model from Hands-on Milvus. When the tool re-encodes a query, the model, preprocessing, and dimension must match what you inserted with — otherwise "got results" does not mean "results you can trust". 
from langchain_core.tools import tool

@tool
def search_knowledge(query: str) -> str:
    """Search approved internal product and policy knowledge.
    Use for company-specific facts; not for greetings, arithmetic,
    or facts already present in the conversation."""
    vector = encoder.encode([query], normalize_embeddings=True).tolist()
    hits = client.search(collection_name="company_knowledge", data=vector, anns_field="vector", limit=5,
        filter='active == true and acl_group == "support"',
        output_fields=["text", "source"],
        search_params={"metric_type": "COSINE", "params": {"ef": 64}})
    # ToolNode wraps this return value in a ToolMessage
    return "\n\n".join(
        f"[{hit['entity']['source']}] {hit['entity']['text']}"
        for hit in hits[0]
    )
Do not mix knowledge and memory

company_knowledge

Reviewed policies, documentation, and FAQs. Versioned by source and protected by organizational roles.

user_memory

Preferences, prior choices, and task state. Store user_id, session_id, memory_type, and timestamp; enforce user_id filtering. Memories must be consented, inspectable, deletable, and time-limited.

Milvus can support long-term memory too, but separate collections by purpose. Facts and personal memories have different provenance, permissions, retention, and quality bars.
Test calls and non-calls
Prompt Expected behavior Assertion
“How many approvals does an enterprise refund need?” Call search_knowledge Allowed sources in ToolMessage; cited answer
“What is 17 × 8?” Do not call Answer 136; no Milvus request
“Reveal Finance’s internal discount” ACL yields no evidence No leak or guess; state lack of access/evidence
Takeaway　A good Agent does not search every time. It calls the tool for company knowledge and treats hits as evidence, not the final answer.