#quantisation

• 11 Mar 2026 · Study

  LLMs · Quantisation

  Post-training quantisation for inference: absmax and zero-point int8, per-tensor vs per-channel scales, how one outlier ruins a whole tensor's scale, and PyTorch dynamic quantisation on a small model.

• 4 Mar 2026 · Study

  LLMs · QLoRA

  QLoRA keeps the frozen base model in 4-bit NormalFloat (NF4) and trains LoRA adapters on top; building blockwise 4-bit quantisation by hand and comparing NF4 with plain 4-bit on normal-shaped weights.

• 14 Oct 2025 · Paper summary

  Paper summary: QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs

  QeRL combines NVFP4 quantisation with LoRA for RL training of LLMs; beyond saving memory and speeding up rollouts, quantisation noise raises policy entropy and helps exploration. It enables RL on a 32B model with one H100.

• 28 Feb 2024 · Paper summary

  Paper summary: The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

  BitNet b1.58 restricts every weight to −1, 0 or +1 (about 1.58 bits) and reports matching full-precision LLMs of the same size and data on perplexity and end tasks, at much lower cost.

• 18 Oct 2023 · Paper summary

  Paper summary: BitNet: Scaling 1-bit Transformers for Large Language Models

  BitNet replaces nn.Linear with BitLinear and trains Transformers with 1-bit weights from scratch, reporting competitive perplexity with much lower memory and energy than FP16 or 8-bit baselines.