Paper summary
Paper summary: MinT: Managed Infrastructure for Training and Serving Millions of LLMs
MinT is infrastructure for training and serving many LoRA policies on a few shared base-model deployments, moving only adapters through rollout, update, evaluation and serving, scaled past 1T-parameter bases and toward million-adapter catalogues.
- Instead of saving a merged full checkpoint per policy, the base model stays loaded and only small LoRA adapters move around.
- Rank-1 adapters can be under 1% of the base model's size.