Paper summary: On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
This position-style study argues that small adapters on top of shared foundation models can serve as persistent personal state (preferences, skills, tool habits, memory-like updates), organised along Scale Up, Scale Down and Scale Out axes.
Problem
PEFT methods like LoRA are usually seen as a cheaper alternative to full fine-tuning. Is there a bigger role for them?
Method
The paper, from the same group as MinT (May 2026 on this blog), treats small trainable adapters as persistent local state on top of strong shared foundation models. The base supplies shared competence, and adapters carry instance-specific preferences, skills, tool habits and memory-like updates. Three scaling axes:
- Scale Up: stronger shared priors make small local updates more useful
- Scale Down: how small adapters can be while staying reliable
- Scale Out: many persistent adapted instances coexisting
MinT is presented as one way to manage adapter identity, revisions, provenance, evaluation and serving.
Key results
From the abstract: together, the results suggest PEFT can be a compact substrate for persistent personal models, not just a budget substitute.
Limitations
There’s no dedicated limitations section in the extracted text. The paper is a framing with supporting studies rather than one benchmark result.
Sources
- On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters (arXiv:2606.02437) — arXiv
- Hugging Face Daily Papers, 2026-06-02 — Hugging Face
- On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters (arXiv HTML full text) — arXiv
This is a summary written from the linked public sources. Figures and numbers are as reported there.