Sahaj.blogs
17 Mar 2025 · Paper summary

Paper summary: SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion

SmolDocling is a 256M-parameter vision-language model that converts whole document pages end to end into DocTags, a markup capturing content, structure and location of every element.

Figure from the paper "SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion"
Figure from Nassar et al. (2025), licensed CC BY 4.0. Figure 1: SmolDocling/SmolVLM architecture. SmolDocling converts images of document pages to DocTags sequences. First, input images are encoded using a vision encoder and reshaped via projection and pooling. Then, the …

Problem

Converting documents (PDF pages, scans) into structured text usually relies on either huge foundation models or pipelines of specialised models (layout detection, OCR, table recognition…), each with its own failure modes.

Method

SmolDocling is a 256M-parameter vision-language model built on SmolVLM (Figure 1 shows the architecture). It reads a page image and generates DocTags, a new markup format that records every page element (text, tables, code, equations, charts, lists) along with its location and context. The authors also contribute new public datasets for charts, tables, equations and code recognition.

Key results

From the abstract: robust reproduction of code listings, tables, equations, charts and lists across business documents, academic papers, technical reports, patents and forms. It competes with VLMs up to 27× larger, with much lower compute.

Limitations

There’s no dedicated limitations section in the extracted text. At the time of the abstract, the datasets were announced as “available soon”.

#multimodal#documents#small-models

Sources

  1. SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion (arXiv:2503.11576) — arXiv
  2. Hugging Face Daily Papers, 2025-03-17 — Hugging Face
  3. SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion (arXiv HTML full text) — arXiv

This is a summary written from the linked public sources. Figures and numbers are as reported there.