#architectures

• 25 Aug 2025 · Study

  Computer Vision · CNN architectures: VGG and ResNet

  VGG's stacks of 3×3 convolutions, why very deep plain networks got worse, and ResNet's skip connections, with a residual block checked in code.

• 15 Aug 2025 · Study

  Computer Vision · CNN architectures: LeNet to AlexNet

  The two early milestones, LeNet-5 for digits and AlexNet for ImageNet: their layers, parameter counts, and what changed between them.

• 8 Oct 2024 · Paper summary

  Paper summary: Differential Transformer

  Differential attention computes attention as the difference between two softmax maps, cancelling common-mode noise; Diff Transformer is reported to beat Transformers on scaling, long context, retrieval and hallucination.

• 1 Apr 2024 · Paper summary

  Paper summary: Jamba: A Hybrid Transformer-Mamba Language Model

  Jamba interleaves Transformer and Mamba layers and adds mixture-of-experts in some layers; the released configuration fits on one 80GB GPU and handles contexts up to 256K tokens.

• 4 Dec 2023 · Paper summary

  Paper summary: Mamba: Linear-Time Sequence Modeling with Selective State Spaces

  Mamba makes state-space model parameters depend on the input, so the model can choose what to remember or forget, and pairs this with a hardware-aware scan: linear-time sequence modelling without attention.