Paper summary
Paper summary: Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
phi-3-mini is a 3.8B model trained on 3.3T tokens of heavily filtered web and synthetic data, reported to reach 69% on MMLU while being small enough to run on a phone.
- The phi recipe again bets on data quality: heavily filtered public web data plus synthetic data.
- The report also introduces 7B and 14B variants and a phi-3.5 series, including an MoE and a vision model.