KNOW THE LIMITS
Trade-Offs & Best Practices
CONCERN
MITIGATION
Router may miss a needed tool
Fallback: raise K, or let the LLM request more
Adds complexity: vector DB + tuning
Embedding search is ms-fast; the savings dominate
Rare tools may rank low
Log misses; retrain or add keyword boosting
K threshold is hard to calibrate
Start at K=5; tune on a dev eval set
Not worth it below ~20 tools
For small tool sets, load statically; no router