#fine-tuning
- 4 Mar 2026 · Study
LLMs · QLoRA
QLoRA keeps the frozen base model in 4-bit NormalFloat (NF4) and trains LoRA adapters on top; building blockwise 4-bit quantisation by hand and comparing NF4 with plain 4-bit on normal-shaped weights.
- 25 Feb 2026 · Study
LLMs · Fine-tuning with LoRA
LoRA freezes the pretrained weights and learns a low-rank update BA; a LoRA layer written from scratch, then PEFT applied to GPT-2 to count how few parameters are trained.
- 23 Dec 2025 · Study
Transformers · Hugging Face Transformers: fine-tuning
Fine-tuning a small pretrained encoder (BERT-mini) for sentiment on Rotten Tomatoes with a plain PyTorch loop, and an honest comparison with the TF-IDF baseline from my NLP phase.
- 23 Jun 2025 · Paper summary
Paper summary: Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
Drag-and-Drop LLMs train a generator that maps a few unlabelled task prompts straight to LoRA weights, specialising an LLM for a new task in seconds without any gradient-based fine-tuning on it.
- 21 Mar 2024 · Paper summary
Paper summary: LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
LlamaFactory is an open framework that brings many efficient fine-tuning methods together for 100+ LLMs, including a no-code web UI called LlamaBoard.
- 22 Sept 2023 · Paper summary
Paper summary: LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
LongLoRA extends the context window of pretrained LLMs cheaply by fine-tuning with shifted sparse local attention plus a LoRA variant that also trains embeddings and norms.