Paper summary
Paper summary: CodeFusion: A Pre-trained Diffusion Model for Code Generation
CodeFusion generates code with a diffusion model that iteratively denoises a whole program conditioned on the natural-language request, instead of writing tokens left to right.
- Autoregressive code models can't easily revise earlier tokens; diffusion refines the whole program at every step.
- A 75M-parameter diffusion model is reported to match much larger autoregressive systems on top-1 accuracy.