With Ulysses context parallelism
Peak memory (GiB)
Default
FSDP
FSDP + Ulysses
OOM (119)
OOM (7684)
OOM (964)
15.0
Model
Attn act.
Other
Llama 3-8B, 3M tokens, 8xH100
AI text/layout recreation from video frame; verify against source image.