KNN has no training step — it memorises the pantry and answers queries by measuring. All the cost moves to prediction time.
k is a bias-variance dial: k=1 memorises noise (jagged boundary), huge k blurs everything toward the majority class.
Because it is pure distance, unscaled features silently rig every vote — the gram-valued column wins. Standardise first, always.
At production scale exact neighbour search dies; approximate indexes (FAISS, HNSW, Annoy) buy speed with a tiny accuracy tax.
In the Test Kitchen: drop the mystery dish in the overlap zone, then swing k from 1 to 15 and watch the verdict flip.
Don't just read the recipe — taste it. Drag, click and break things below.
Purple dots are samosas, teal dots are dosas, mapped by flavour. Click anywhere to drop a mystery dish — its k nearest neighbours vote on its identity. Try k = 1 near the messy middle (one odd neighbour fools it), then k = 15 (the vote smooths out, but local detail dies).
FIG L.7: K-NEAREST NEIGHBOURS — NO TRAINING, JUST DISTANCE + DEMOCRACY. THE DASHED CIRCLE IS THE ELECTORATE
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
# scaling is NOT optional: KNN is pure distance
knn = make_pipeline(StandardScaler(),
KNeighborsClassifier(n_neighbors=5))
knn.fit(X_train, y_train)
print(knn.score(X_test, y_test))
# odd k avoids ties; sweep k with cross-validation, not vibes