AI updates
PyTorch 2.1: dynamic shapes in torch.compile, distributed checkpointing
PyTorch 2.1 added automatic dynamic-shape support to torch.compile, parallel distributed checkpointing, NumPy compilation, and a prototype of torch.export.
- torch.compile now detects changing tensor shapes and avoids unnecessary recompiles automatically.
- torch.distributed.checkpoint saves and loads from many ranks in parallel and can reshard to a new cluster layout.