M16 / LABORATORY

Training. Loss falls.
A decision boundary forms.

A neural net is just a function. Training is gradient descent on its parameters until the function fits. Watch loss curves, weights, and a real decision boundary settle into place — in real time, on a real (tiny) 2-layer MLP.

EXPERIMENT 09.01 · TRAINING.SIM EPOCH · 0 STEP · 0 LOSS · — ACC · —

DATASET

SAMPLES 160

NOISE σ 0.05

HYPERPARAMETERS

HIDDEN 8

LR 0.10

MOMENTUM 0.90

ACTIVATION

OPTIMIZER

DECISION BOUNDARY · 2-LAYER MLP → live forward pass on a 40×40 grid

CLASS 0

CLASS 1

LOSS

Cross-entropy · train (●) vs eval (◌)

ACCURACY

LEARNING RATE LANDSCAPE

↘ optimizer trajectory on a synthetic 2D loss

// 02

Weights & gradients · what's inside the model

W1 · input → hidden [2 × 8]

W2 · hidden → output [8 × 2]

What you're watching → Each frame computes a forward pass on every grid pixel and backprops on every data point. Crank LR too high → loss explodes. Use Tanh on SPIRAL → boundary stays smooth, takes longer. ReLU on XOR → very sharp boundary, very fast.