M18 / LABORATORY

Representation learning. Good features
without (many) labels.

CLIP and DINO feel exotic, but the engine underneath is old and general: take a perfectly normal backbone — ResNet, VGG — and train it so that similar things get similar vectors. Change only the definition of "similar" and you get contrastive learning, metric learning, or self-supervision. Same backbone, different supervision signal.

EXPERIMENT 12.01 · METRIC.LEARNING BACKBONE · RESNET / VGG (standard) HEAD · MLP PROJECTION → ℓ2-NORM STATUS · LIVE

// 01

The shape of it · backbone + projection head

ANY CNN BECOMES AN ENCODER

→

ResNet / VGG
backbone f(·)

→

global pool
2048-D

→

MLP head g(·)
→ 128-D

→

ℓ2-normalize
z on unit sphere

The only new piece is the loss. The backbone is the exact same ResNet you'd train for ImageNet classification. Instead of a softmax-over-classes head, you bolt on a small projection MLP and train with a similarity objective. After training, you throw the head away and keep the backbone — its features now transfer to almost any downstream task.

WHO DEFINES "SIMILAR"?

SimCLR · two augmentations of the same image are similar. No labels.

Triplet / metric · a labelled anchor is similar to same-class, far from other-class.

DINO · crops should agree with a momentum teacher. No labels.

CLIP · an image is similar to its caption. Weak web labels.

// 02

SimCLR · positives from augmentation, negatives from the batch

Take N images. Make two random augmentations of each → 2N views. The two views of one image are a positive pair; every other view in the batch is a negative. The NT-Xent loss pulls each positive pair together and pushes everything else apart. Identical idea to CLIP's matrix — except both axes are images.

ONE IMAGE · TWO VIEWS

VIEW A

≈

VIEW B

Crop, flip, jitter, blur — different pixels, same content. The encoder must map both to nearly the same z.

NT-XENT

for a positive pair (i, j)

ℓ = −log exp(sᵢⱼ/τ) / Σ_k≠i exp(sᵢₖ/τ)

VIEW-A × VIEW-B SIMILARITY diagonal = positive pairs

TRAIN STEP 0%

// 03

Triplet loss · the labelled cousin

DRAG THE POSITIVE & NEGATIVE anchor is fixed

TRIPLET LOSS

MARGIN α 0.20

d(a, p) · —

d(a, n) · —

loss = max(0, d(a,p) − d(a,n) + α)

loss · —

state · —

Drag the positive closer than the negative by at least the margin to drive the loss to zero. This is how face-recognition embeddings (FaceNet) are trained.

// 04

What you get · an organized embedding space

As any of these losses trains, the embedding space self-organizes: same-class points clump, different classes separate. This is the input to the Embeddings & Latents module — drag the slider and watch a random cloud resolve into clean clusters, exactly what a trained ResNet encoder produces.

EMBEDDING SPACE · projected to 2-D 6 classes

TRAINING

EPOCH 0

silhouette · 0.02

1-NN acc · 18%

The throughline → Classification, contrastive, triplet, CLIP, DINO — all of them are just different answers to "what should be close to what?" Pick the supervision you can afford (labels are expensive, augmentations are free), bolt the right loss onto a standard backbone, and you get transferable features. That's representation learning.