AI updates
PyTorch 2.2: FlashAttention-2 in SDPA, AOTInductor
PyTorch 2.2 brought FlashAttention-2 into scaled_dot_product_attention for about 2× speedups, and introduced AOTInductor for compiling models ahead of time for non-Python deployment.
- The same F.scaled_dot_product_attention call got about 2× faster by switching to FlashAttention-2 underneath.
- AOTInductor compiles a PyTorch program ahead of time so it can run server-side without Python.