Slides: SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI

Source Video

SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI

Relationship To World's Fair 2026

These slides are extracted from a public AI Engineer YouTube video connected to World's Fair 2026. Speaker-matched clips are supporting context unless later confirmed as exact session recordings; official livestream recordings are day-level/event-level source material.

Related Scheduled Sessions

No individual scheduled session mapping has been assigned yet; treat this as an event livestream deck.

Extracted Slides

Recreated text/layout view: open HTML recreation
AI slide classifier: title_card confidence 0.99
Text source: agent_vision.

Slide text:

SWE-Marathon

Can coding agents stay coherent over a 1 billion token budget?

Rishi Desai

abundant

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.98
Text source: advanced OCR rapidocr-live/border-trim/opencv-adaptive.
OCR decision: ready — Multiple small cards and quotes; OCR will read the slide more accurately than manual triage.

Slide text:

Agents are moving to autonomous end-to-end projects

AI anthropic:con/engineer ing Anthropic github.coa/openai/parancter-golf OpenAl

Claudes'. "Building a C compiler with a team of paralle! checkpoint budget: "Parameter Golf" - train a GPT under a tiny

Cloudflare Cursor

olog.cloudttare.coa/vlnext Cursor.con/blog/scsling-agents

"How we rebuilt Next.js with Al in one week: "Scaling long-running autonomous coding

2.1.U

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.99
Text source: advanced OCR rapidocr-live/bright-screen/opencv-adaptive.
OCR decision: ready — Dense benchmark comparison with small labels and token counts; OCR is appropriate.

Slide text:

From coding tasks to engineering projects.

2021 HumanEval Function completion -l nin per TasK Inlocked coot tlns:..IK: ToKENS

2023 SWE-bench -s-is hin per task Real GitHub issues UNLOCKID COOING AGEHTS l0OK TOKEHS:

2025 Terminal-Bench Multi-step terminal tasks -s-is HIn per tasK URloCxCD TEAHInAL AcEHiS. IH TOKENS:

:2026. SWE-Marathon -3-lo Hours peR task Days-long agentic work UhtockihG Autohotous AGthts 1B+: T0KENS

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.98
Text source: advanced OCR rapidocr-live/border-trim/contrast.
OCR decision: ready — Contains a large embedded UI screenshot with smaller text that OCR should handle better than hand transcription.

Slide text:

First benchmark with full-stack CUA-verified tasks

CUA verifier grades Slack clone SWE-MARATHON

A computer-usc agent drives the Ul hikc arcal user CiaudeOpus 47·ClaudeCodc

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.98
Text source: advanced OCR rapidocr-live/border-trim/contrast.
OCR decision: ready — Embedded product UI screenshot with checklist text; OCR is the better capture path.

Slide text:

First benchmark with full-stack CUA-verified tasks

Igeneral

Workspace layout CUAVERIFIER PASS

SuRK

Chec0]/C9

Sgn up & sign n

Sossion/account

Slack-like layout

sve-sarathoa.9rg

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.99
Text source: advanced OCR rapidocr-live/border-trim/opencv-adaptive.
OCR decision: ready — Chart labels and small task lists are OCR-suitable.

Slide text:

20 tasks across four families

Algoritmic 2 tasts Full-stack product clones S tasks

slt-cloee excelclcre s3-cttee.

Library cloees tasks

MLE Library clones & reproductions 8 tasks

tals. kubernetes-rust-reurite

. rust-c-coapiler.

syn s Product clonts - jax-pytorch-rewrite MLE 5 tasks

post-train-ifeval

.u.

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.99
Text source: advanced OCR rapidocr-live/bright-screen/opencv-adaptive.
OCR decision: ready — Dense leaderboard table and small chart labels are better handled by OCR.

Slide text:

Leaderboard

1,400 trlls 31.3M try' tobens / triat:iongest trit 877.4M

Al Ciaude Opus 4.8 / ctabe Ctoe HODEl / AGEHT ResotutIoN Rate ipAssei1 26.0%

Al Cisude Opus 4.7 / ctuce Cooe 16.0%

7. GLM 5.2 / (1** Co4s 13.0%

GpT-5.5 / (oden Ct1 12.0%

Gemiru 3.5 Flash / Ce=inl (l1 7.0%

G DeepStek V4 Pro / Ttralnus 2 4.0%

Gemini 31 Pro Preview / Ge=int Ct1 2.0%.

MiniMax M2.7 / Teratrat? 0.0%

Kimi K2.6 / x1ml (ode (t1 0.0%

452 734

.7 10:

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.99
Text source: advanced OCR rapidocr-live/bright-screen/opencv-adaptive.
OCR decision: ready — Timeline, metrics, and chart annotations are dense and OCR-suitable.

Slide text:

A 356M token rollout for nextjs-vite-rewrite

9.4 hours agonr walciock Rrpo + faturt ti plor stlon. 844 trsiectory steps imgiemantation push P+O+ rOAt + SSR 842total tool acticns Hydration brek throughr: 220/325 ptting tdrt/nrite: 223 (26%) buidtet: 323 (38%) (41) 1l:oqox,bnqsp rtsd/esrch: 259 (31%) other t0ols: 26 (3%)

1.3h 2.9h 6.1a 7.0h $.5h 岁.33

Acticns per 30 min 83 Frst tl suile. 0325 p+sting Tarsron dtbugging. misng boottp midataware Cusler Server sctions + tspcche:pena rluL 283325

68

40

2 3 Elapsed agont wall-ciock timo (hours) 5 6 8 Z cu s.2 / (1

$/ 13.

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.99
Text source: advanced OCR rapidocr-live/bright-screen/opencv-adaptive.
OCR decision: ready — Bar chart and small labels are OCR-suitable.

Slide text:

Reward hacking

12.8% 9.2%

susplciois shortcut behuvior Clesr tiploit shlpped earees re-ard via'eigteit

HODel / aGfh! Shart or taials (<)

Gemini 3.1 Pro Provlew / Cealol Cl1 30.0%

Gpt-5.5 / Cotex Cl1 19.0%

Geminu 3.5 Flash / Geainl Cl1 16.0%

Kiml X2.6 / Kla1 Code Ctl 10.0%

Al Ciaude Opus 4.8 / Ctaudt Code %1'6

DopScok V4 Pro / Itrnihus? 6.3%

Al Claude Opus 4.7 / (luode Code 3.0%

7. GLM 5.2 / Clde Code 3.0%

。 MiniMax M2.7 / tereirus? 1.0%

41 21

细 Susplc lo+s tortrut huvier 1ctesr trplelt saipotl

1tru.

Recreated text/layout view: open HTML recreation
AI slide classifier: content_slide confidence 0.99
Text source: advanced OCR rapidocr-live/border-trim/opencv-adaptive.
OCR decision: ready — Code sample, comparison boxes, and audit text are dense and OCR-suitable.

Slide text:

Gemini 3.1 Pro cheats on the C compiler task

RUST-C-CONPILER

Task: build a C compiler in Rust:UNIT TESTS. 0.989

ExPecteO implement lexer, parser. codegen in Rust use gcc inside Rust code SUBnItiED SHoRrCUr. Looks almost solved if you only compare output.

:ANTI-CHEAT AUDIT

std::process::Cormand::new( gcc) cargs([P, “ input, cstatus() “o% output]) The verifier runs'strace, sees gcc spawn, and rejects compilor wrappers. 0.989 partial: +. 0 reward

S++-+a?at h++.tf9 11./ 11

Classification audit: raw/sources/slide-ai-classification/slides/Rx8f05JI_WA/audit.json

Slide-Derived Subjects To Review

Subject extraction uses video title, related session titles/descriptions, transcript context, and OCR text when available. OCR is best-effort and should be reviewed against the embedded slide images.