AI updates
vLLM 0.9: PyTorch 2.7, Blackwell kernels, large-scale inference
vLLM 0.9.0 moved to PyTorch 2.7 and CUDA 12.8, added initial optimised kernels for NVIDIA Blackwell, and started data-parallel, expert-parallel and prefill/decode disaggregation support for large-scale inference.
- Expert parallelism and prefill/decode disaggregation are the building blocks for serving very large MoE models across many GPUs.
- The release had 649 commits from 215 contributors.