Weight Folding, CUDA Streams, and the Bug That Made My Model Speak Backwards
Official Schedule Context
- Date/time: 2026-07-01 · 3:45pm-4:05pm
- Track/room: Inference · Track 9
- Speaker(s): Filip Makraduli
- Session type/status: session · confirmed
Official Description
A talk about contributing GPU benchmarks to an open-source research paper (FlashNorm). I'll walk
through the engineering journey: folding norm weights into projections, writing Triton kernels,
accidentally making attention bidirectional (oops), and ultimately proving a 33-35% speedup on the
norm+project operation. Practical lessons for anyone trying to optimize transformer inference.
Related YouTube Video
The Small Model Infrastructure Nobody Built (So We Did) — Filip Makraduli, Superlinked (speaker-match related prior/adjacent AI Engineer video; captions: English auto-captions).
Transcript Status
Related video transcript availability: English auto-captions. Treat this as supporting context, not a recording of this exact scheduled session unless later confirmed. Not fetched yet.
People
Notes
- Pending transcript synthesis when an official recording or confirmed matching video is available.
Supporting Slides
- youtube qdh_x uRs9g slides — extracted from the related public AI Engineer video.