2 hr deep dive on LLM Inference at Scale — Part 1 of 2

Official Schedule Context

Most engineers using LLMs can call an API. Far fewer can explain why their model is slow, why it's

running out of memory, or how the inference engines powering every major LLM API actually work. This

workshop walks through the full inference stack — from how a transformer generates a single token to

serving billions of tokens a day with vLLM, SGLang, TensorRT-LLM, Ray, and KServe/llm-d. 60%

explanation with live demos, 40% hands-on exercises. Attendees leave with a running vLLM server they

benchmarked themselves. Based on the open-source practitioners handbook being built live at

github.com/harshuljain13/llm-inference-at-scale (NOTE: this is a 2 hour workshop that happens over

lunch break - you should try to have lunch before or after if attending) compute kindly sponsored

by Coreweave/Marimo!

No official session recording transcript was found by exact title match on the AI Engineer YouTube channel during this run.

Pending transcript synthesis when an official recording or confirmed matching video is available.