Gemma 4 12B: an encoder-free multimodal model for laptops
Google introduced Gemma 4 12B, a mid-sized open model where vision and audio inputs flow straight into the LLM backbone with no separate encoders; it runs in 16GB and ships with multi-token-prediction drafters.
What happened
Google DeepMind introduced Gemma 4 12B (the page is dated 3 June 2026), between the edge-friendly E4B and the larger 26B MoE. The post says Gemma 4 models had crossed 150 million downloads.
The technical details
From the post:
- unified architecture with no multimodal encoders: vision and audio inputs flow directly into the LLM backbone
- the first mid-sized Gemma with native audio input
- benchmark performance nearing the 26B model, for multi-step reasoning and agentic workflows
- runs locally in 16GB of VRAM or unified memory
- Apache 2.0
- ships with Multi-Token Prediction (MTP) drafters for lower latency through speculative decoding
Why it matters
Most multimodal models I studied (CLIP, ViT-based VLMs) bolt a separate encoder onto the language model. Dropping the encoder is the “early fusion” direction (Chameleon, Emu3, Llama 4) reaching a laptop-sized open model.
#llm#multimodal#on-device#google
Sources
- Introducing Gemma 4 12B: a unified, encoder-free multimodal model — Google DeepMind
This is a summary written from the linked public sources. Figures and numbers are as reported there.