Sahaj.blogs
8 Apr 2025 · Paper summary

Paper summary: SmolVLM: Redefining small and efficient multimodal models

SmolVLM is a family of compact vision-language models designed for on-device inference; the 256M model uses under 1GB of GPU memory and is reported to beat the 300× larger Idefics-80B.

Figure from the paper "SmolVLM: Redefining small and efficient multimodal models"
Figure from Marafioti et al. (2025), licensed CC BY 4.0. Figure 1 : Smol yet Mighty: comparison of SmolVLM with other state-of-the-art small VLM models. Image results are sourced from the OpenCompass OpenVLM leaderboard ( Duan et al., 2024 ) .

Problem

Big VLMs need a lot of compute. Small VLMs often copy big models’ design choices, especially extensive image tokenisation, which wastes memory and makes them impractical on devices.

Method

The authors systematically explore architecture configurations, tokenisation strategies and data curation designed for low compute, and identify the design choices that give the biggest gains for image and video tasks at small sizes. Figure 1 compares SmolVLM with other small VLMs using OpenCompass leaderboard results.

Key results

From the abstract:

  • SmolVLM-256M uses under 1GB of GPU memory at inference and outperforms the 300× larger Idefics-80B, released 18 months earlier
  • the largest model, 2.2B, rivals state-of-the-art VLMs that use twice the GPU memory
  • the models also show robust video understanding

Limitations

There’s no dedicated limitations section in the extracted text. Figure 1’s comparison uses OpenCompass leaderboard results, as the paper notes.

#multimodal#small-models#efficiency

Sources

  1. SmolVLM: Redefining small and efficient multimodal models (arXiv:2504.05299) — arXiv
  2. Hugging Face Daily Papers, 2025-04-08 — Hugging Face
  3. SmolVLM: Redefining small and efficient multimodal models (arXiv HTML full text) — arXiv

This is a summary written from the linked public sources. Figures and numbers are as reported there.