Batch Size 1 Bottleneck: Why Your Diffusion Model Needs CUDA Graphs, Not Distillation

Developers often pursue model distillation to accelerate diffusion inference, but profiling reveals the real bottleneck: GPU idleness caused by kernel launch overhead and memory traffic. Simple optimizations—torch.compile with CUDA graphs, fused attention backends, and CFG batching—can deliver 30-40% latency improvements before any architectural changes become necessary.