M13 / ARCHITECTURES

GANs. A forger and
a detective, locked in a duel.

Two networks play a game. The Generator forges samples from noise; the Discriminator tries to tell real from fake. Each makes the other better. When the forger wins, its fakes are indistinguishable from reality. This is a real GAN — both networks train live, in your browser, with full backpropagation.

EXPERIMENT 08.01 · ADVERSARIAL.GAME STEP · 0 D(real) · — D(fake) · —

// 01

The adversarial game

GENERATOR vs DISCRIMINATOR

noise z ~ 𝒩(0,1)

2-D latent

→

GENERATOR G

2→16→2 · tanh

"the forger"

→

DISCRIMINATOR D

2→16→1 · sigmoid

"the detective"

→

p(real)

0 … 1

the minimax objective

min_G max_D 𝔼_x~real[log D(x)] + 𝔼_z~noise[log(1 − D(G(z)))]

Read it as a duel → D wants to maximize the expression: push D(real)→1 and D(fake)→0. G wants to minimize it: fool D into outputting D(fake)→1. Neither can win permanently — every improvement by one creates pressure on the other. Equilibrium is reached when fakes are statistically identical to reals and D is stuck guessing 0.5.

THE TRAINING STEP

Sample real points + generate fakes. Train D to label reals 1, fakes 0. (detective studies)

Freeze D. Generate fakes, backprop the signal "make D say 1" through D into G's weights. (forger adapts)

The gradient that updates G flows through the frozen discriminator — that's the only way G ever learns what "realistic" means. G never sees a real sample directly.

// 02

Live training playground

TARGET DISTRIBUTION

HYPERPARAMETERS

G LR 0.020

D LR 0.020

D STEPS / G 1

BATCH 128

SHOW D-SURFACE

SAMPLE SPACE ● real ● fake (G output)

D=0 (fake)

D=1 (real)

LOSSES

— D loss — G loss

DIAGNOSTICS

D(real) → —

D(fake) → —

coverage → —

state → init

When D(real)≈D(fake)≈0.5 the generator has won — D can no longer tell them apart.

// 03

Mode collapse · the classic GAN failure

WHEN THE FORGER GETS LAZY

Select the 8 MODES target above and train. Watch what often happens: the generator discovers one mode that reliably fools the discriminator, and dumps all its samples there — ignoring the other seven clusters entirely.

This is mode collapse. The generator minimizes its loss without capturing the full diversity of the data. It found a local cheat: the detective can't punish it fast enough for ignoring modes it isn't currently looking at.

Fixes in practice: minibatch discrimination, unrolled GANs, Wasserstein loss (WGAN-GP), spectral normalization, or simply more D steps per G step — try cranking D STEPS / G to 3–5 and watch coverage improve.

Healthy: fakes (magenta) blanket every real cluster. coverage → 8/8

Collapsed: fakes pile onto 1–2 clusters. coverage → 2/8

The coverage readout in the diagnostics panel counts how many target modes have generated samples nearby.

Why GANs matter → Before diffusion models, GANs were how machines learned to generate — faces, art, super-resolution, image-to-image translation. The adversarial idea (one network supervising another) reappears everywhere: from domain adaptation to RLHF reward models. This 2D toy is the same math that drove StyleGAN.