Product Requirements
System Design
Evaluation and Monitoring
Production Readiness
Identify the business problem
Identify your constraints
Define the role of AI
Define success metrics
Identify data sources and retrieval techniques
Select system architecture and tech stack
Determine the UX and feedback loops
Define system guardrails
Define offline evaluation metrics
Define online evaluation metrics
Optimize for accuracy
Optimize for cost and latency
Optimize for reliability