AIE
What Makes a Good Task?
Problem description
1. A good task balances clarity
2. Too easy and too hard both fail
3. Complexity: breadth or depth
Reliable verifier
1. Should reward actual fixes, should reject fake solutions
2. Not too narrow, not too wide
Stable infrastructure is part of eval
1. Minimal infra noise during runs
2. Connection might blink, images might become stale, pipeline might break (1970s bug)
1. A good task balances clarity
2. Too easy and too hard both fail
3. Complexity: breadth or depth
Reliable verifier
1. Should reward actual fixes, should reject fake solutions
2. Not too narrow, not too wide
Stable infrastructure is part of eval
1. Minimal infra noise during runs
2. Connection might blink, images might become stale, pipeline might break (1970s bug)
AI Engineer EUROPE