---
title: "Slides: SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI"
category: "slides"
video_id: "Rx8f05JI_WA"
sourceLabels: ["Public YouTube video frames", "Public YouTube metadata"]
---

# Slides: SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI

## Source Video
[SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI](https://www.youtube.com/watch?v=Rx8f05JI_WA)

## Relationship To World's Fair 2026
These slides are extracted from a public AI Engineer YouTube video connected to World's Fair 2026. Speaker-matched clips are supporting context unless later confirmed as exact session recordings; official livestream recordings are day-level/event-level source material.

## Related Scheduled Sessions
- No individual scheduled session mapping has been assigned yet; treat this as an event livestream deck.

## Extracted Slides
![[assets/slides/Rx8f05JI_WA/slide-001.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-001.html)
- AI slide classifier: `title_card` confidence `0.99`
- Text source: agent_vision.

Slide text:

> SWE-Marathon
> Can coding agents stay coherent over a 1 billion token budget?
> Rishi Desai
> abundant

![[assets/slides/Rx8f05JI_WA/slide-002.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-002.html)
- AI slide classifier: `content_slide` confidence `0.98`
- Text source: advanced OCR `rapidocr-live/border-trim/opencv-adaptive`.
- OCR decision: ready — Multiple small cards and quotes; OCR will read the slide more accurately than manual triage.

Slide text:

> Agents are moving to autonomous end-to-end projects
> AI anthropic:con/engineer ing Anthropic github.coa/openai/parancter-golf OpenAl
> Claudes'. "Building a C compiler with a team of paralle! checkpoint budget: "Parameter Golf" - train a GPT under a tiny
> Cloudflare Cursor
> olog.cloudttare.coa/vlnext Cursor.con/blog/scsling-agents
> "How we rebuilt Next.js with Al in one week*: "Scaling long-running autonomous coding*
> 2.1.U

![[assets/slides/Rx8f05JI_WA/slide-003.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-003.html)
- AI slide classifier: `content_slide` confidence `0.99`
- Text source: advanced OCR `rapidocr-live/bright-screen/opencv-adaptive`.
- OCR decision: ready — Dense benchmark comparison with small labels and token counts; OCR is appropriate.

Slide text:

> From coding tasks to engineering projects.
> 2021 HumanEval Function completion -l nin per TasK Inlocked coot tlns:..IK: ToKENS
> 2023 SWE-bench -s-is hin per task Real GitHub issues UNLOCKID COOING AGEHTS l0OK TOKEHS:
> 2025 Terminal-Bench Multi-step terminal tasks -s-is HIn per tasK URloCxCD TEAHInAL AcEHiS. IH TOKENS:
> :2026. SWE-Marathon -3-lo Hours peR task Days-long agentic work UhtockihG Autohotous AGthts 1B+: T0KENS

![[assets/slides/Rx8f05JI_WA/slide-004.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-004.html)
- AI slide classifier: `content_slide` confidence `0.98`
- Text source: advanced OCR `rapidocr-live/border-trim/contrast`.
- OCR decision: ready — Contains a large embedded UI screenshot with smaller text that OCR should handle better than hand transcription.

Slide text:

> First benchmark with full-stack CUA-verified tasks
> CUA verifier grades Slack clone SWE-MARATHON
> A computer-usc agent drives the Ul hikc arcal user CiaudeOpus 47·ClaudeCodc

![[assets/slides/Rx8f05JI_WA/slide-005.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-005.html)
- AI slide classifier: `content_slide` confidence `0.98`
- Text source: advanced OCR `rapidocr-live/border-trim/contrast`.
- OCR decision: ready — Embedded product UI screenshot with checklist text; OCR is the better capture path.

Slide text:

> First benchmark with full-stack CUA-verified tasks
> Igeneral
> Workspace layout CUAVERIFIER PASS
> SuRK
> Chec0]/C9
> Sgn up & sign n
> Sossion/account
> Slack-like layout
> sve-sarathoa.9rg

![[assets/slides/Rx8f05JI_WA/slide-006.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-006.html)
- AI slide classifier: `content_slide` confidence `0.99`
- Text source: advanced OCR `rapidocr-live/border-trim/opencv-adaptive`.
- OCR decision: ready — Chart labels and small task lists are OCR-suitable.

Slide text:

> 20 tasks across four families
> Algoritmic 2 tasts Full-stack product clones S tasks
> slt-cloee excelclcre s3-cttee.
> Library cloees tasks
> MLE Library clones & reproductions 8 tasks
> tals. kubernetes-rust-reurite
> . rust-c-coapiler.
> syn s Product clonts - jax-pytorch-rewrite MLE 5 tasks
> post-train-ifeval
> .u.

![[assets/slides/Rx8f05JI_WA/slide-007.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-007.html)
- AI slide classifier: `content_slide` confidence `0.99`
- Text source: advanced OCR `rapidocr-live/bright-screen/opencv-adaptive`.
- OCR decision: ready — Dense leaderboard table and small chart labels are better handled by OCR.

Slide text:

> Leaderboard
> 1,400 trlls 31.3M try' tobens / triat:iongest trit 877.4M
> Al Ciaude Opus 4.8 / ctabe Ctoe HODEl / AGEHT ResotutIoN Rate ipAssei1 26.0%
> Al Cisude Opus 4.7 / ctuce Cooe 16.0%
> 7. GLM 5.2 / (1** Co4s 13.0%
> GpT-5.5 / (oden Ct1 12.0%
> Gemiru 3.5 Flash / Ce=inl (l1 7.0%
> G DeepStek V4 Pro / Ttralnus 2 4.0%
> Gemini 31 Pro Preview / Ge=int Ct1 2.0%.
> MiniMax M2.7 / Teratrat? 0.0%
> Kimi K2.6 / x1ml (ode (t1 0.0%
> 452 734
> .7 10:

![[assets/slides/Rx8f05JI_WA/slide-008.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-008.html)
- AI slide classifier: `content_slide` confidence `0.99`
- Text source: advanced OCR `rapidocr-live/bright-screen/opencv-adaptive`.
- OCR decision: ready — Timeline, metrics, and chart annotations are dense and OCR-suitable.

Slide text:

> A 356M token rollout for nextjs-vite-rewrite
> 9.4 hours agonr walciock Rrpo + faturt ti plor stlon. 844 trsiectory steps imgiemantation push P+O+ rOAt + SSR 842total tool acticns Hydration brek throughr: 220/325 ptting tdrt/nrite: 223 (26%) buidtet: 323 (38%) (41) 1l:oqox,bnqsp rtsd/esrch: 259 (31%) other t0ols: 26 (3%)
> 1.3h 2.9h 6.1a 7.0h $.5h 岁.33
> Acticns per 30 min 83 Frst tl suile. 0325 p+sting Tarsron dtbugging. misng boottp midataware Cusler Server sctions + tspcche:pena rluL 283325
> 68
> 40
> 2 3 Elapsed agont wall-ciock timo (hours) 5 6 8 Z cu s.2 / (1
> $/ 13.

![[assets/slides/Rx8f05JI_WA/slide-009.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-009.html)
- AI slide classifier: `content_slide` confidence `0.99`
- Text source: advanced OCR `rapidocr-live/bright-screen/opencv-adaptive`.
- OCR decision: ready — Bar chart and small labels are OCR-suitable.

Slide text:

> Reward hacking
> 12.8% 9.2%
> susplciois shortcut behuvior Clesr tiploit shlpped earees re-ard via'eigteit
> HODel / aGfh! Shart or taials (<)
> Gemini 3.1 Pro Provlew / Cealol Cl1 30.0%
> Gpt-5.5 / Cotex Cl1 19.0%
> Geminu 3.5 Flash / Geainl Cl1 16.0%
> Kiml X2.6 / Kla1 Code Ctl 10.0%
> Al Ciaude Opus 4.8 / Ctaudt Code %1'6
> DopScok V4 Pro / Itrnihus? 6.3%
> Al Claude Opus 4.7 / (luode Code 3.0%
> 7. GLM 5.2 / Clde Code 3.0%
> 。 MiniMax M2.7 / tereirus? 1.0%
> 41 21
> 细 Susplc lo+s tortrut huvier 1ctesr trplelt saipotl
> 1tru.

![[assets/slides/Rx8f05JI_WA/slide-010.jpg]]

- Recreated text/layout view: [open HTML recreation](/assets/slide-recreations/slides/Rx8f05JI_WA/slide-010.html)
- AI slide classifier: `content_slide` confidence `0.99`
- Text source: advanced OCR `rapidocr-live/border-trim/opencv-adaptive`.
- OCR decision: ready — Code sample, comparison boxes, and audit text are dense and OCR-suitable.

Slide text:

> Gemini 3.1 Pro cheats on the C compiler task
> RUST-C-CONPILER
> Task: build a C compiler in Rust:UNIT TESTS. 0.989
> ExPecteO implement lexer, parser. codegen in Rust use gcc inside Rust code SUBnItiED SHoRrCUr. Looks almost solved if you only compare output.
> :ANTI-CHEAT AUDIT
> std::process::Cormand::new( gcc) cargs([P, “ input, cstatus() “o% output]) The verifier runs'strace, sees gcc spawn, and rejects compilor wrappers. 0.989 partial: +. 0 reward
> S++-+a?at h++.tf9 11./ 11


Classification audit: `raw/sources/slide-ai-classification/slides/Rx8f05JI_WA/audit.json`

## Slide-Derived Subjects To Review
Subject extraction uses video title, related session titles/descriptions, transcript context, and OCR text when available. OCR is best-effort and should be reviewed against the embedded slide images.