Markdown source

Slides: 20 days of compute vs 7 hours: rethinking what state-of-the-art means — Bertrand Charpentier, Pruna

Source Video

20 days of compute vs 7 hours: rethinking what state-of-the-art means — Bertrand Charpentier, Pruna

Relationship To World's Fair 2026

These slides are extracted from a public AI Engineer YouTube video connected to World's Fair 2026. Speaker-matched clips are supporting context unless later confirmed as exact session recordings; official livestream recordings are day-level/event-level source material.

Related Scheduled Sessions

• No individual scheduled session mapping has been assigned yet; treat this as an event livestream deck.

Extracted Slides

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.99
• Text source: agent_vision.

Slide text:

Method 1: Check Public Leaderboards

Use-case: Find the most performant image editing AI model

- Step 1: Find a leaderboard

- Step 2: Select the top ranked model

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.98
• Text source: advanced OCR rapidocr-live/border-trim/contrast.
• OCR decision: ready — small leaderboard screenshots and labels are better handled by OCR

Slide text:

Problem 1: Each public leaderboard has a different ranking

AIE

(LM)Arena Design Arena Artificial Analysis

Engineering the future of Al

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.98
• Text source: advanced OCR rapidocr-live/right-72/opencv-adaptive.
• OCR decision: ready — chart text and small labels are better handled by OCR

Slide text:

Problem 3:Leaderboard are not statistically significant for one use case

Artificial Analysis ranking w! number of samples Win rate on (LM)Arena Win rate on Design Arena

AlEngineer

EUROPE

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.99
• Text source: agent_vision.

Slide text:

Method 2: Perform Internal Evaluation

Use-case: Find the most performant image editing AI model

- Step 1.a: Manual inspection

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.89
• Text source: agent_vision.
• OCR decision: ready — Dense chart labels and small benchmark graphics are better handled by OCR.

Slide text:

Problem 5: Benchmark results are not consistent

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.97
• Text source: advanced OCR rapidocr-live/border-trim/contrast.
• OCR decision: ready — Multiple charts plus small bullet text make OCR the cheaper and more accurate path.

Slide text:

Problem 5: Benchmark results are not consistent

AIE

Misaligned with target Benchmarkranking change Smal/largevariations Checkmeaningsofmetrics Use multiple metrics

Engineering the future of Al

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.98
• Text source: agent_vision.
• OCR decision: ready — Scatter plots and small axis labels are OCR-suitable.

Slide text:

What Model is State-of-the-Art?

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.97
• Text source: advanced OCR rapidocr-live/bright-screen/contrast.
• OCR decision: ready — The benchmark scatter plots and annotations are small enough that OCR will likely outperform manual transcription.

Slide text:

What Model is State-of-the-Art?

There are multiple SOTA Al models

AIE Long Text Bench OeelGTertRenderieg

0:20

0.85

0.03

0.73

0.65 0.63

0.50 0.00

0.55 0.55

Medin inference Time Mesinn IeterenceTime

AI Engineer

EUROPE

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.97
• Text source: advanced OCR rapidocr-live/border-trim/contrast.
• OCR decision: ready — Endpoint cards and small brand/logo text are OCR-suitable.

Slide text:

How To Reach SOTA Performance?

Option 1:Use optimized Performance models

AIE

VideoEnipoints ImageEndpoints Text Endpoints AudioEndpoints

Bopioote Koyed aws Runware Segmind WIRO

Engineering the future of Al

neer

• Recreated text/layout view: open HTML recreation
• AI slide classifier: content_slide confidence 0.97
• Text source: advanced OCR rapidocr-live/bright-screen/contrast.
• OCR decision: ready — This slide has multiple small cards, labels, and links that are best left for OCR.

Slide text:

How To Reach SOTA Performance?

Option 1: Use optimized Performance models

AIE

VideoEndpoints Image Endpoints Text Endpoints AudioEndpoints

Weplicoto Koyeb aws Runware Segmnd WIRO

Option 2: Develop and apply inference optimization

PrunoAI

AI EfficiencyPackage AlEfficiencyMaterials AI Efficiency Courses

aRaihub.comPnnaAwcoun

AI Engineer

JUUU

Hidden Non-Slide Evidence

• `slide-001.jpg` — speaker_stage confidence 0.98; camera shot of speaker on stage with projected slide; not a clean slide frame
• `slide-002.jpg` — title_card confidence 0.96; title slide only; no substantive content

Classification audit: raw/sources/slide-ai-classification/slides/hqHC6Z_lXyo/audit.json

Slide-Derived Subjects To Review

Subject extraction uses video title, related session titles/descriptions, transcript context, and OCR text when available. OCR is best-effort and should be reviewed against the embedded slide images.