Study notes

My topic-by-topic notes, each with a small runnable example. See them in order on the learning path.

• 23 Sept 2025 · Study

  Computer Vision · Data augmentation

  Label-preserving random transforms (flips, crops, rotation, colour jitter) with torchvision.transforms.v2, applied consistently to images and boxes.

• 16 Sept 2025 · Study

  Computer Vision · Segmentation and U-Net

  Semantic segmentation as per-pixel classification, the U-Net encoder–decoder with skip connections, Dice/IoU, and a tiny U-Net trained on synthetic shapes.

• 8 Sept 2025 · Study

  Computer Vision · Object detection and YOLO

  Detection as boxes plus classes, IoU, non-maximum suppression, YOLO's single-pass grid idea, and running a pretrained torchvision detector on a NASA photo.

• 1 Sept 2025 · Study

  Computer Vision · Transfer learning

  Reusing an ImageNet-pretrained ResNet-18: frozen features plus a new linear head versus training from scratch, on the 8×8 digits upsampled.

• 25 Aug 2025 · Study

  Computer Vision · CNN architectures: VGG and ResNet

  VGG's stacks of 3×3 convolutions, why very deep plain networks got worse, and ResNet's skip connections, with a residual block checked in code.

• 15 Aug 2025 · Study

  Computer Vision · CNN architectures: LeNet to AlexNet

  The two early milestones, LeNet-5 for digits and AlexNet for ImageNet: their layers, parameter counts, and what changed between them.

• 8 Aug 2025 · Study

  Computer Vision · OpenCV essentials

  The OpenCV operations I use most: colour conversion, resizing, Gaussian blur, Canny edges, thresholding and contours.

• 1 Aug 2025 · Study

  Computer Vision · Image basics

  Images as arrays: height × width × channels, uint8 vs float, RGB vs grayscale, cropping, and the channel-order and range traps between libraries.

• 23 Jul 2025 · Study

  NLP · Text classification

  Multi-class topic classification on 20 Newsgroups, comparing Naive Bayes, logistic regression and a linear SVM, with a per-class report and the most confused pairs.

• 14 Jul 2025 · Study

  NLP · Sentiment analysis

  Classifying movie-review sentiment on the Rotten Tomatoes dataset: a TF-IDF plus logistic regression baseline, its most telling words, and where it fails.

• 4 Jul 2025 · Study

  NLP · Named entity recognition

  NER as token-level sequence labelling with BIO tags, running spaCy's pretrained pipeline, and why entity spans make evaluation tricky.

• 25 Jun 2025 · Study

  NLP · Attention (Bahdanau)

  Additive attention lets the decoder look at every encoder state instead of one summary vector; the same reversal task with attention, and its alignment weights.

• 17 Jun 2025 · Study

  NLP · Seq2seq

  The encoder–decoder architecture for sequence-to-sequence tasks, teacher forcing, and a small GRU model learning to reverse digit strings.

• 6 Jun 2025 · Study

  NLP · GloVe

  GloVe vectors from global co-occurrence statistics, loading pretrained 50-dimensional vectors, analogies, and the bias these vectors inherit.

• 28 May 2025 · Study

  NLP · word2vec

  Dense word vectors learned by predicting context (skip-gram), trained with gensim on the 20 Newsgroups corpus, with nearest neighbours.

• 20 May 2025 · Study

  NLP · Bag of words and TF-IDF

  Turning documents into vectors with CountVectorizer and TfidfVectorizer, n-grams, and finding similar documents with cosine similarity.

• 9 May 2025 · Study

  NLP · Tokenisation

  Word, character and subword tokenisation compared, out-of-vocabulary words, and what a real BERT WordPiece tokenizer does to unusual words.

• 1 May 2025 · Study

  NLP · Text preprocessing

  Cleaning raw text with regex and spaCy: normalisation, lowercasing, stop words, lemmatisation, and why modern models need much less of this.

• 22 Apr 2025 · Study

  Deep Learning · LSTM and GRU

  Gated recurrent units: the LSTM's cell state and three gates, the simpler GRU, and an experiment measuring how far gradients reach back through 100 time steps.

• 11 Apr 2025 · Study

  Deep Learning · RNN

  Recurrent networks carry a hidden state across time steps; the unrolled view, nn.RNN shapes, and why gradients vanish over long sequences.

• 3 Apr 2025 · Study

  Deep Learning · CNN intro

  Convolutions as sliding filters with shared weights, padding, stride, pooling, output-size arithmetic, and a small CNN versus an MLP on digits.

• 25 Mar 2025 · Study

  Deep Learning · PyTorch training loop

  The standard PyTorch loop with Dataset, DataLoader, nn.Module, a loss, an optimiser and an evaluation pass, applied to scikit-learn's handwritten digits.

• 14 Mar 2025 · Study

  Deep Learning · PyTorch tensors and autograd

  PyTorch tensors (NumPy-like, with devices and dtypes), the dynamic computation graph, requires_grad, backward(), and no_grad.

• 6 Mar 2025 · Study

  Deep Learning · Dropout

  Dropout randomly zeroes activations during training as a regulariser; inverted scaling, train vs eval behaviour, and its effect on an overfitting MLP.

• 25 Feb 2025 · Study

  Deep Learning · Batch normalisation

  Normalising each feature over the mini-batch, the learnable scale and shift, train vs eval mode, and its effect on a deep MLP.

• 17 Feb 2025 · Study

  Deep Learning · Weight initialisation

  Why initial weight scale matters, Xavier/Glorot for tanh and He/Kaiming for ReLU, and watching activations explode or collapse through 50 layers.

• 6 Feb 2025 · Study

  Deep Learning · Optimisers: SGD, momentum, Adam

  Mini-batch SGD, momentum, RMSProp-style scaling and Adam compared on an ill-conditioned valley, using torch.optim.

• 28 Jan 2025 · Study

  Deep Learning · Activation functions

  Sigmoid, tanh, ReLU, Leaky ReLU and GELU compared by shape and gradient, and why saturating activations make deep networks hard to train.

• 20 Jan 2025 · Study

  Deep Learning · Backpropagation

  Backpropagation as the chain rule applied layer by layer; a two-layer network trained on XOR in pure NumPy, plus a numerical gradient check.

• 9 Jan 2025 · Study

  Deep Learning · Multilayer perceptron

  Adding a hidden layer with a non-linear activation turns the perceptron into an MLP that can solve XOR; a forward pass written in NumPy.

• 1 Jan 2025 · Study

  Deep Learning · The perceptron

  Rosenblatt's perceptron from scratch: a weighted sum, a step function, and the update rule; it learns AND but can never learn XOR.

• 23 Dec 2024 · Study

  Machine Learning · Hyperparameter search

  Tuning with GridSearchCV and RandomizedSearchCV, nested evaluation on a held-out test set, and why random search often wins.

• 13 Dec 2024 · Study

  Machine Learning · Feature engineering

  Creating better inputs: ratios, date parts, binning, interactions, log transforms for skewed values, and target-free encodings.

• 5 Dec 2024 · Study

  Machine Learning · scikit-learn pipelines

  Combining preprocessing and model with Pipeline and ColumnTransformer for mixed numeric and categorical data, so one object is fitted, tuned and saved.

• 27 Nov 2024 · Study

  Machine Learning · Cross-validation

  k-fold, stratified and grouped cross-validation, why one train/test split is noisy, and avoiding leakage by preprocessing inside the folds.

• 19 Nov 2024 · Study

  Machine Learning · Model evaluation metrics

  Confusion matrix, accuracy and why it fails on imbalanced data, precision, recall, F1, ROC-AUC, and regression metrics.

• 11 Nov 2024 · Study

  Machine Learning · PCA

  Principal component analysis: rotating data onto directions of maximum variance, explained variance ratio, and compressing the digits dataset.

• 1 Nov 2024 · Study

  Machine Learning · Hierarchical clustering and DBSCAN

  Two alternatives to k-means: agglomerative clustering, which builds a tree of merges, and DBSCAN, which finds dense regions of any shape and labels noise.

• 24 Oct 2024 · Study

  Machine Learning · k-means clustering

  Unsupervised clustering with k-means: the assign/update loop, inertia, choosing k with the elbow and silhouette score.

• 16 Oct 2024 · Study

  Machine Learning · Support vector machines

  SVMs find the widest-margin boundary; support vectors, the C parameter, and the kernel trick for non-linear data.

• 8 Oct 2024 · Study

  Machine Learning · Boosting and XGBoost

  Boosting builds shallow trees one after another, each fixing the previous errors; gradient boosting, learning rate, early stopping, and XGBoost.

• 30 Sept 2024 · Study

  Machine Learning · Random forest

  Bagging many decorrelated trees: bootstrap samples, random feature subsets, out-of-bag score, and feature importances.

• 20 Sept 2024 · Study

  Machine Learning · Decision trees

  How a decision tree chooses splits using Gini impurity, why deep trees overfit, and reading the learned rules.

• 12 Sept 2024 · Study

  Machine Learning · k-nearest neighbours

  k-NN classifies by majority vote of the closest training points; why scaling matters and how k controls the bias–variance trade-off.

• 4 Sept 2024 · Study

  Machine Learning · Regularisation: ridge and lasso

  Overfitting with too many features, and how L2 (ridge) shrinks weights while L1 (lasso) drives some exactly to zero.

• 27 Aug 2024 · Study

  Machine Learning · Logistic regression

  Logistic regression for binary classification: the sigmoid, log-loss, decision thresholds, and predicted probabilities.

• 19 Aug 2024 · Study

  Machine Learning · Gradient descent

  Fitting linear regression by gradient descent from scratch: the gradient of MSE, the learning rate, and what happens when it is too small or too big.

• 9 Aug 2024 · Study

  Machine Learning · Linear regression

  Linear regression with many features: the model, the squared-error cost, the normal equation, and scikit-learn's LinearRegression with RMSE and R².

• 1 Aug 2024 · Study

  Machine Learning · The ML workflow

  The end-to-end supervised learning loop: define the task, split data, pick a baseline, fit, evaluate on held-out data, iterate.

• 25 Jul 2024 · Study

  Probability and Statistics · Simple linear regression

  Fitting y = a + bx by least squares, reading slope and intercept, R², residuals, and the bridge to machine learning.

• 18 Jul 2024 · Study

  Probability and Statistics · Correlation

  Covariance, Pearson vs Spearman correlation, what r does and doesn't capture, and why correlation is not causation.

• 11 Jul 2024 · Study

  Probability and Statistics · t-test and chi-square

  One-sample, two-sample (Welch) and paired t-tests, and the chi-square test of independence for categorical data.

• 5 Jul 2024 · Study

  Probability and Statistics · Hypothesis tests

  The logic of a hypothesis test: null and alternative, test statistic, p-value, significance level, and the two types of error.

• 28 Jun 2024 · Study

  Probability and Statistics · Point and interval estimation

  Estimators and their bias, standard error, and building and interpreting 95% confidence intervals with the t distribution.

• 21 Jun 2024 · Study

  Probability and Statistics · Sampling

  Population vs sample, simple random vs stratified sampling, sampling bias, and the bootstrap for estimating uncertainty.

• 17 Jun 2024 · Study

  Probability and Statistics · Central Limit Theorem

  Why sample means look normal even when the data doesn't, the σ/√n standard error, and a simulation from a skewed distribution.

• 10 Jun 2024 · Study

  Probability and Statistics · Continuous distributions

  Uniform, normal and exponential distributions, z-scores, the 68–95–99.7 rule, and computing areas with cdf and ppf.

• 3 Jun 2024 · Study

  Probability and Statistics · Discrete distributions

  Bernoulli, binomial, geometric and Poisson distributions: what each one counts, their means, and computing them with scipy.stats.

• 27 May 2024 · Study

  Probability and Statistics · Random variables

  Random variables as numbers attached to outcomes, PMF vs PDF vs CDF, expectation and variance, and linearity of expectation.

• 21 May 2024 · Study

  Probability and Statistics · Bayes' theorem

  Bayes' theorem as updating a prior with evidence, the base-rate trap in medical tests, and sequential updating.

• 14 May 2024 · Study

  Probability and Statistics · Conditional probability

  P(A | B) as shrinking the sample space to B, the multiplication rule, the law of total probability, and the Monty Hall problem by simulation.

• 7 May 2024 · Study

  Probability and Statistics · Probability rules

  Sample spaces and events, the complement, addition and multiplication rules, independence, and checking them by simulation.

• 1 May 2024 · Study

  Probability and Statistics · Descriptive statistics

  Summarising data with centre (mean, median, mode) and spread (variance, standard deviation, IQR), and why the median resists outliers.

• 25 Apr 2024 · Study

  Matplotlib · Mini EDA project

  A small end-to-end exploratory analysis of the Palmer Penguins dataset with pandas and Seaborn: shape, missing values, distributions, groups and relationships.

• 19 Apr 2024 · Study

  Matplotlib · Seaborn statistical plots

  Seaborn on top of Matplotlib: plots driven by DataFrame columns, hue for groups, and the statistical plots (box, violin, regression, heatmap).

• 16 Apr 2024 · Study

  Matplotlib · Styling

  Making plots readable: style sheets, colours, labels with units, annotations, removing chart junk, and saving at the right DPI.

• 10 Apr 2024 · Study

  Matplotlib · Subplots

  Grids of plots with plt.subplots, shared axes, uneven layouts with GridSpec / subplot_mosaic, and constrained layout.

• 4 Apr 2024 · Study

  Matplotlib · Line, bar, scatter and histogram

  The four plots I'll use most and when each fits: line for trends, bar for categories, scatter for relationships, histogram for distributions.

• 1 Apr 2024 · Study

  Matplotlib · Figure and Axes

  Matplotlib's object model: the Figure as the canvas and Axes as one plot, and why the explicit fig, ax = plt.subplots() style is clearer.

• 27 Mar 2024 · Study

  Pandas · Time series

  DatetimeIndex, date_range, the .dt accessor, resampling to weekly or monthly, rolling windows and shift.

• 25 Mar 2024 · Study

  Pandas · pivot_table

  Reshaping between long and wide formats with pivot_table, melt and crosstab, including margins and fill_value.

• 21 Mar 2024 · Study

  Pandas · merge and join

  SQL-style joins with merge (inner, left, right, outer), the indicator flag, validate, and stacking with concat.

• 18 Mar 2024 · Study

  Pandas · groupby

  Split-apply-combine with groupby: aggregations, named agg, multiple keys, transform vs agg, and filter.

• 14 Mar 2024 · Study

  Pandas · Missing data

  Detecting NaN with isna, counting missing values per column, dropna vs fillna, filling with group means, and interpolation.

• 12 Mar 2024 · Study

  Pandas · Cleaning data

  Fixing messy columns: stripping and normalising strings, converting types with to_numeric, renaming, dropping duplicates and mapping categories.

• 7 Mar 2024 · Study

  Pandas · Selection with loc and iloc

  Label-based loc versus position-based iloc, boolean filtering, query(), and avoiding chained assignment.

• 5 Mar 2024 · Study

  Pandas · Reading and writing files

  read_csv and to_csv with the parameters that matter (sep, usecols, dtype, parse_dates, index=False), plus JSON and Parquet.

• 1 Mar 2024 · Study

  Pandas · Series and DataFrame

  Series (a labelled 1-D array) and DataFrame (a table of Series sharing an index), and how index alignment works.

• 27 Feb 2024 · Study

  NumPy · Random numbers

  The Generator API (default_rng), seeding for reproducibility, distributions, shuffling, and sampling without replacement.

• 22 Feb 2024 · Study

  NumPy · Linear algebra

  Matrix multiplication with @, solving linear systems, inverse vs solve, determinants, norms, and eigen decomposition.

• 20 Feb 2024 · Study

  NumPy · Vectorisation

  Replacing Python loops with whole-array operations and ufuncs, with a timing comparison and the axis argument for reductions.

• 15 Feb 2024 · Study

  NumPy · Broadcasting

  The broadcasting rules that let arrays of different shapes combine, and three uses: centring columns, outer products, and pairwise distances.

• 12 Feb 2024 · Study

  NumPy · Boolean and fancy indexing

  Selecting with boolean masks and integer arrays, combining conditions with & and |, np.where, and why these return copies.

• 8 Feb 2024 · Study

  NumPy · Indexing and slicing

  Multi-dimensional indexing with a[i, j], slicing rows and columns, and the most important rule: basic slices are views, not copies.

• 5 Feb 2024 · Study

  NumPy · dtypes

  NumPy dtypes (int32, int64, float32, float64, bool), memory per element, casting with astype, and silent integer overflow.

• 1 Feb 2024 · Study

  NumPy · ndarray basics

  The ndarray: creating arrays, shape, ndim, size, reshape, and why one NumPy array beats a list of floats.

• 26 Jan 2024 · Study

  C++ · DSA: linked list

  A singly linked list built with unique_ptr nodes: push_front, traversal, in-place reversal, and how it compares with vector.

• 23 Jan 2024 · Study

  C++ · DSA: sorting

  Insertion sort and merge sort written by hand, compared with std::sort and std::stable_sort on the same data.

• 18 Jan 2024 · Study

  C++ · DSA: binary search

  Binary search by hand with the overflow-safe midpoint, then std::lower_bound, upper_bound and binary_search.

• 15 Jan 2024 · Study

  C++ · DSA: stack and queue

  LIFO stacks and FIFO queues with std::stack and std::queue, plus two classic uses: bracket matching and BFS.

• 10 Jan 2024 · Study

  C++ · Smart pointers

  unique_ptr for single ownership, shared_ptr with reference counting, weak_ptr to break cycles, and make_unique/make_shared.

• 5 Jan 2024 · Study

  C++ · Memory: new, delete and RAII

  Stack vs heap, manual new/delete and why it leaks, and RAII: tying a resource's lifetime to an object's scope.

• 2 Jan 2024 · Study

  C++ · STL: algorithms

  The <algorithm> and <numeric> toolkit: sort with comparators, find_if, count_if, accumulate, transform, and min/max.

• 27 Dec 2023 · Study

  C++ · STL: set

  std::set and std::unordered_set for unique elements, insert's return value, and set operations with algorithms.

• 22 Dec 2023 · Study

  C++ · STL: map and unordered_map

  Ordered std::map (a balanced tree) versus std::unordered_map (a hash table), lookups with find/count, and the operator[] side effect.

• 19 Dec 2023 · Study

  C++ · STL: vector

  std::vector as C++'s list: push_back, size vs capacity, iterators, erase, and reserving memory.

• 14 Dec 2023 · Study

  C++ · Templates

  Function and class templates, how the compiler stamps out one version per type, and how this differs from Python's duck typing.

• 11 Dec 2023 · Study

  C++ · OOP: inheritance and virtual functions

  Public inheritance, virtual functions and runtime polymorphism, override, pure virtual (abstract) classes, and virtual destructors.

• 6 Dec 2023 · Study

  C++ · OOP: classes and constructors

  Classes with public/private members, constructors with initialiser lists, const member functions, and destructors.

• 1 Dec 2023 · Study

  C++ · Functions and overloading

  Declaring vs defining functions, default arguments, overloading by parameter types, and lambdas.

• 27 Nov 2023 · Study

  C++ · std::string

  std::string versus C strings, common operations (substr, find, append, compare), mutability, and converting numbers.

• 22 Nov 2023 · Study

  C++ · Arrays

  C-style arrays, std::array, 2D arrays, out-of-bounds risk, and why .at() exists.

• 17 Nov 2023 · Study

  C++ · References

  References as aliases, pass-by-value vs pass-by-reference, and const references for passing big objects cheaply.

• 14 Nov 2023 · Study

  C++ · Pointers

  What a pointer is, & and * operators, nullptr, pointer arithmetic over an array, and why dangling pointers are dangerous.

• 9 Nov 2023 · Study

  C++ · Control flow

  if/else, switch, the three kinds of for loop, while and do-while, written next to their Python equivalents.

• 6 Nov 2023 · Study

  C++ · Variables, types and I/O

  Static types in C++, sizes of the built-in types, integer overflow, auto, const, and reading input with cin.

• 1 Nov 2023 · Study

  C++ · Basics: compiling and hello world

  Starting C++: the compile-then-run model, main(), #include, and what g++ actually produces.

• 27 Oct 2023 · Study

  Python · venv and pip

  Isolating project dependencies with venv, installing with pip, and pinning versions in requirements.txt.

• 24 Oct 2023 · Study

  Python · Decorators

  Functions as objects, closures, writing decorators with functools.wraps, and decorators that take arguments.

• 19 Oct 2023 · Study

  Python · Generators

  Generator functions with yield, lazy evaluation, infinite sequences, pipelines, and itertools.

• 16 Oct 2023 · Study

  Python · Iterators

  The iterator protocol behind every for loop: iter(), next(), StopIteration, and writing an iterator class.

• 11 Oct 2023 · Study

  Python · OOP: inheritance and dunder methods

  Subclassing with super(), method overriding, and dunder methods (__repr__, __eq__, __len__, __add__) that plug objects into Python syntax.

• 6 Oct 2023 · Study

  Python · OOP: classes and objects

  Defining classes, __init__ and self, instance vs class attributes, methods, and properties.

• 3 Oct 2023 · Study

  Python · Exceptions

  try/except/else/finally, catching specific exceptions, raising your own, and custom exception classes.

• 29 Sept 2023 · Study

  Python · Files: reading and writing

  Opening files with with-blocks, text vs binary modes, encodings, reading line by line, and pathlib.

• 26 Sept 2023 · Study

  Python · Modules and imports

  How import finds modules, the different import forms, packages with __init__.py, and the if __name__ == "__main__" guard.

• 21 Sept 2023 · Study

  Python · Comprehensions

  List, dict and set comprehensions, filtering with if, nested loops, and when a generator expression is the better choice.

• 18 Sept 2023 · Study

  Python · Functions: *args, **kwargs and scope

  Variable-length arguments with *args and **kwargs, keyword-only parameters, unpacking at the call site, and LEGB scope.

• 13 Sept 2023 · Study

  Python · Functions: arguments and return values

  Defining functions, positional vs keyword arguments, default values, the mutable-default trap, and returning several values.

• 8 Sept 2023 · Study

  Python · Control flow: if, for, while

  if/elif/else, for loops over iterables with range, enumerate and zip, while loops, and break/continue/else.

• 5 Sept 2023 · Study

  Python · Sets

  Sets for uniqueness and fast membership tests, plus union, intersection and difference.

• 1 Sept 2023 · Study

  Python · Dictionaries

  Dictionaries for key-value lookups: get() with defaults, iterating items(), insertion order, and counting with Counter.

• 29 Aug 2023 · Study

  Python · Tuples and unpacking

  Tuples as fixed records, the one-element tuple comma, unpacking with *rest, and swapping without a temp variable.

• 24 Aug 2023 · Study

  Python · Lists

  Lists as mutable sequences: append vs extend, slicing, sorting in place vs sorted(), and the aliasing trap.

• 21 Aug 2023 · Study

  Python · Strings: f-strings and formatting

  f-strings, the format mini-language (width, precision, alignment, thousands separators) and the = debugging shortcut.

• 16 Aug 2023 · Study

  Python · Strings: indexing, slicing, methods

  Strings as immutable sequences: indexing, negative indexes, slicing with steps, and the common methods.

• 11 Aug 2023 · Study

  Python · Types: int, float, bool, None

  Python's basic scalar types, why 0.1 + 0.2 is not exactly 0.3, and how None differs from False.

• 8 Aug 2023 · Study

  Python · Syntax: indentation, comments, print

  How Python uses indentation instead of braces, how comments work, and the options print() takes.

• 4 Aug 2023 · Study

  Python · Setup: installing Python and running a script

  Day one: installing Python 3, checking the version, and running a first script from the terminal.