Paper summary: Scaling Synthetic Data Creation with 1,000,000,000 Personas

Problem

LLM-generated synthetic data tends to be repetitive. Asking for “a maths problem” a million times gives many near-duplicates. Diversity is the bottleneck.

Method

Add a persona to each generation prompt (“as a marine biologist…”, “as a truck driver…”), so the LLM draws on a different perspective each time. To do this at scale, the authors build Persona Hub: 1 billion personas curated automatically from web data, about 13% of the world’s population. They describe personas as distributed carriers of world knowledge.

Key results

The paper demonstrates persona-driven synthesis for:

• mathematical and logical reasoning problems
• instructions (user prompts)
• knowledge-rich texts
• game NPCs
• tools (function definitions)

The authors argue it is versatile, scalable, flexible and easy to use.

Limitations

There’s no dedicated limitations section in the extracted text. The evidence is largely in the form of use-case demonstrations. The initial public release is a subset: 200,000 of the personas plus samples of the synthesised data (e.g. 50,000 maths problems and 50,000 instructions).