#transformers

• 26 Jan 2026 · AI updates

  Transformers v5: the first major release in five years

  Hugging Face released Transformers v5, the first major version in five years, removing long-deprecated APIs, refactoring weight loading and tokenization, and moving to weekly minor releases.

• 23 Dec 2025 · Study

  Transformers · Hugging Face Transformers: fine-tuning

  Fine-tuning a small pretrained encoder (BERT-mini) for sentiment on Rotten Tomatoes with a plain PyTorch loop, and an honest comparison with the TF-IDF baseline from my NLP phase.

• 12 Dec 2025 · Study

  Transformers · Hugging Face Transformers: pipelines

  The pipeline() API for quick inference (sentiment, zero-shot classification, NER, summarisation) and what happens inside: tokenizer → model → post-processing.

• 4 Dec 2025 · Study

  Transformers · Vision Transformer (ViT)

  ViT treats an image as a sequence of 16×16 patches with a [CLS] token and position embeddings; patch arithmetic, and a pretrained ViT classifying a photo.

• 25 Nov 2025 · Study

  Transformers · GPT

  GPT as a decoder-only Transformer trained to predict the next token; generating with GPT-2, measuring perplexity, and seeing why causal masking makes this possible.

• 17 Nov 2025 · Study

  Transformers · BERT

  BERT as a bidirectional encoder pretrained with masked language modelling; running fill-mask, looking at [CLS] and [SEP], and seeing contextual embeddings change with meaning.

• 6 Nov 2025 · Study

  Transformers · Encoder and decoder

  How the encoder layer, the decoder layer and cross-attention fit together, with residual connections and layer norm, and the three families they led to: encoder-only, decoder-only and encoder–decoder.

• 28 Oct 2025 · Study

  Transformers · Positional encoding

  Why attention needs position information, the sinusoidal encoding from the original paper, learned position embeddings, and the relative-position idea behind RoPE.

• 20 Oct 2025 · Study

  Transformers · Multi-head attention

  Splitting d_model into several heads so attention can look at different things at once, implemented by reshaping, and checked against nn.MultiheadAttention.

• 9 Oct 2025 · Study

  Transformers · Self-attention

  Scaled dot-product self-attention from scratch: queries, keys, values, the √d scaling, softmax weights, and a causal mask, checked against PyTorch's built-in function.

• 1 Oct 2025 · Study

  Transformers · Attention Is All You Need

  Reading the 2017 Transformer paper: dropping recurrence entirely, building everything from attention and feed-forward layers, and why that parallelises so well.

• 9 Aug 2024 · Paper summary

  Paper summary: Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation

  Transformer Explainer is an interactive, in-browser visual tool that runs a live GPT-2 so non-experts can follow a Transformer's data flow and watch next-token predictions change as they edit input and settings.

• 17 May 2024 · AI updates

  Transformers 4.41: GGUF support, Phi-3, PaliGemma

  Transformers v4.41 added Phi-3 (with 128K-context variants), the JetMoE mixture-of-experts model, Google's PaliGemma vision-language model and GGUF loading support.

• 2 Nov 2023 · AI updates

  Transformers 4.35: safetensors by default, Distil-Whisper, Fuyu

  Hugging Face Transformers v4.35 made safetensors the default serialization format and added models including Distil-Whisper (6× faster than Whisper) and Adept's Fuyu-8B.