Machine Learning · k-means clustering

This is my first unsupervised algorithm: there are no labels, just structure to find.

k-means:

1. Pick k starting centres (scikit-learn uses the smarter k-means++ initialisation).
2. Assign every point to its nearest centre.
3. Update each centre to the mean of its assigned points.
4. Repeat until nothing changes.

It minimises inertia, the within-cluster sum of squared distances. Inertia always falls as k grows, so to choose k, look for an elbow or use the silhouette score (−1 to 1, higher is better).

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score, adjusted_rand_score

X, y_true = make_blobs(n_samples=600, centers=4, cluster_std=0.9, random_state=7)

for k in range(2, 8):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
    print(f"k={k}: inertia {km.inertia_:8.1f} | silhouette {silhouette_score(X, km.labels_):.3f}")

best = KMeans(n_clusters=4, n_init=10, random_state=0).fit(X)
print("agreement with true blobs (ARI):", round(adjusted_rand_score(y_true, best.labels_), 3))
print("centres:\n", best.cluster_centers_.round(2))

Output:

k=2: inertia  18162.5 | silhouette 0.594
k=3: inertia   5018.2 | silhouette 0.773
k=4: inertia    913.2 | silhouette 0.830
k=5: inertia    826.1 | silhouette 0.696
k=6: inertia    745.7 | silhouette 0.571
k=7: inertia    663.4 | silhouette 0.467
agreement with true blobs (ARI): 1.0
centres:
 [[ 9.54  0.72]
 [-8.5   5.58]
 [ 0.21 -8.68]
 [-1.22  4.27]]