Study
Transformers · Hugging Face Transformers: fine-tuning
Fine-tuning a small pretrained encoder (BERT-mini) for sentiment on Rotten Tomatoes with a plain PyTorch loop, and an honest comparison with the TF-IDF baseline from my NLP phase.
- Fine-tuning reuses everything the model learned in pre-training and only has to learn the new head and small adjustments.
- An 11M-parameter BERT-mini only tied the TF-IDF baseline here: model size matters, and strong classical baselines are hard to beat on short texts.