Paper summary: MinT: Managed Infrastructure for Training and Serving Millions of LLMs

Problem

A common pattern: many trained policies (personalised or task-specific models) on top of a few expensive base-model deployments. Storing and serving a full merged checkpoint per policy doesn’t scale.

Method

MinT (MindLab Toolkit) keeps the base model resident and moves exported LoRA adapter revisions through rollout, update, export, evaluation, serving and rollback, hiding the distributed systems work behind a service interface. It scales along three axes:

• Scale Up: LoRA RL on frontier-scale dense and MoE models (including MLA and DSA attention), validated beyond 1T total parameters
• Scale Down: move only adapters, under 1% of base size at rank 1
• Scale Out: separate which policies are addressable from which are loaded on GPUs, for very large catalogues

Key results

From the abstract:

• adapter-only handoff cuts the measured step by 18.3× (4B dense) and 2.85× (30B MoE)
• concurrent multi-policy GRPO cuts wall time by 1.77× and 1.45× without raising peak memory
• 10^6-scale addressable catalogues (single-engine sweeps measured to 100K) and thousand-adapter active waves
• packed MoE LoRA tensors speed live loading by 8.5–8.7×

Limitations

There’s no dedicated limitations section in the extracted text. It’s a systems paper, and quality depends on what LoRA can express for each policy.