Paper summary
Paper summary: Scaling Latent Reasoning via Looped Language Models
Ouro builds latent reasoning into pre-training with looped language models (iterating in latent space with a learned depth), and its 1.4B and 2.6B models are reported to match up to 12B models.
- Instead of reasoning only through generated chain-of-thought text, the model loops its layers in latent space, and this is trained from pre-training onward.
- The authors attribute the gains to better knowledge manipulation, not more stored knowledge.