What is an Inference Engine, Anyway?

Official Schedule Context

Official Description

To run state-of-the-art inference yourself, you must master the inference engine: vLLM, SGLang, TRT-

LLM, or your own jawn. The inference engine manages the lifecycle of an inference request, from

input to output. In this workshop, we'll examine the architecture of modern high performance

inference engines, the key techniques that inference engines need to deliver that performance, and

the traces and metrics that inference engines emit.

Related YouTube Video

How fast are LLM inference engines anyway? — Charles Frye, Modal (speaker-match related prior/adjacent AI Engineer video; captions: English auto-captions).

Transcript Status

Related video transcript availability: English auto-captions. Treat this as supporting context, not a recording of this exact scheduled session unless later confirmed. Not fetched yet.

People

Notes

Supporting Slides

Slide Evidence