EVAL #008: NVIDIA Just Open-Sourced an Inference Engine. Now What?

At GTC 2026, NVIDIA open-sourced Dynamo, a Rust-and-Python inference orchestration framework that disaggregates prefill and decode phases onto separate GPU pools, connected via zero-copy RDMA KV cache transfers. This directly challenges existing serving engines like vLLM and TensorRT-LLM by introducing Kubernetes-like orchestration for LLM inference at data center scale. The same week saw major releases from PyTorch 2.7, vLLM, SGLang, and Transformers, making it the most consequential week for AI inference tooling in recent memory.