Slides: Evals Are Broken, Use Them Anyway — Ara Khan, Cline
Source Video
Evals Are Broken, Use Them Anyway — Ara Khan, Cline
Relationship To World's Fair 2026
These slides are extracted from a public AI Engineer YouTube video connected to World's Fair 2026. Speaker-matched clips are supporting context unless later confirmed as exact session recordings; official livestream recordings are day-level/event-level source material.
Related Scheduled Sessions
- No individual scheduled session mapping has been assigned yet; treat this as an event livestream deck.
Extracted Slides
OCR text:
Evals Are Broken,Use Tneni Anyway
AlEngine
OCR text:
a | want you to be able to build, interpret and/or use
. * evals in your agent flows
: " Y Engineering the future of Al
» 4 r
OCR text:
B Wee eure Cas
ra ae n The new model from Meta is already looking like a disappointment:
bl bd overoptimized for public benchmark numbers at the detriment of
vt * i. everything else. Knowing how to evaluate models in a way that correlates
with actual usefulness is a core competency for Al labs, and any new lab is
unlikely to be successful without first figuring that out.
| AlEngineer |
: ~ EUROPE
/ a
OCR text:
ro Nikunj Kothari |
Genuinely curious - has any engineer made a decision on a model (or
harness) based on a benchmark result?
a Every researcher / engineer | have talked to routinely dismisses it. They
Penn trust their taste, evals and how models are performing for their specific use
" * Cee
od ve *
It’s seems like it’s simply now used for the labs to show how they’re slightly
ahead but in my head has no material impact on whether a model is used or
not.
I'd love to be proven wrong so please push back!
i
| Al Engineer |
W009
OCR text:
eee
ve Why SWE-bench Verified
aoe no longer measures
frontier coding capabilities
Engineering the future of Al
OCR text:
terminal-bench: benchmarks for ai
yoX—) Leg 3 aes TON ea
Pd Pa
[nr to tst oy age
Engineering the future of Al
, Pl
OCR text:
ra eee
es i g* “- =
PA a + 4
rt1's ais ¥
mee
Was
OCR text:
ek ee ———F '* .- .
i i :
—_— wy | : .
7 ee poopy tt ha :
[ on ii
oor WEL SY eee
rea arcu Boe
ve aah Cl teak vy
BNET rT
i | r
; r
a PA ;
a 2 i | ,
% ——
a w
~ ¥ oo:
_. . re :
4 : md ,
_ .
Slide-Derived Subjects To Review
Subject extraction uses video title, related session titles/descriptions, transcript context, and OCR text when available. OCR is best-effort and should be reviewed against the embedded slide images.