THE HARD PART

The hardest problem wasn't retrieval. It was knowing when retrieval was wrong.

LLM-based scoring
Asked the model to rate relevance. Accurate but +2-3s latency and cost per query.

Fixed thresholds
cosine > 0.7 = relevant. Broke on short queries and long queries alike.

Simple heuristic won
50% similarity + 30% keyword + 20% recency. Adaptive. 0.4ms, no API calls.

Confidence scoring blend

Similarity 50%
Keywords 30%
Recency 20%