Humans struggled to tell these images apart: on MNIST, they guessed at chance; on CIFAR‑10, they were wrong about one in five. The surprising part isn’t the pictures—it’s the training tricks that finally made a famously unstable method behave.

The adversarial game that wouldn’t settle

Generative adversarial networks (GANs) are a two‑player game. A generator proposes fake images; a discriminator tries to spot them. The goal is a Nash equilibrium: a state where neither player can improve by changing its strategy alone. In practice, the “game” is non‑convex and high‑dimensional, and ordinary gradient descent can circle endlessly rather than settle. Researchers know the failure mode by sight: mode collapse, when the generator emits near‑identical images, chasing whatever the current discriminator thinks looks real. Losses may oscillate, samples look repetitive, and the game never stabilizes.

This paper from OpenAI offers a toolkit that nudges the game toward stability—not by one grand fix, but by several small, targeted changes that reduce chasing and encourage diversity.

Two stabilizers: match features and look across the batch

The first move reframes the generator’s job. Instead of directly trying to “fool” the discriminator, the generator matches statistics of real data computed inside the discriminator. Concretely, pick an intermediate layer f(·) in the discriminator and train the generator to minimize the gap between the average features of real images and the average features of generated ones: ||E_data[f(x)] − E_z[f(G(z))]||². This “feature matching” gives the generator a moving but sensible target. It stops overfitting to the discriminator’s current quirks and reduces violent swings.

The second move gives the discriminator something it was missing: a sense of the group. With minibatch discrimination, the discriminator learns to measure how similar each example is to others in the same batch. If the generator collapses and produces clones, the discriminator sees the cluster and penalizes it. Under the hood, the model projects per‑image features through a learnable tensor and adds new features that sum up distances to other samples in the batch. The result is immediate: diversity rises, and samples look better to humans.

These two tricks complement each other. Feature matching shines when the goal is a strong classifier that can learn from unlabeled data. Minibatch discrimination shines when the goal is striking, varied images.

Small asymmetries that matter

  • One‑sided label smoothing: Replace “real” labels of 1 with a softer value like 0.9, but keep “fake” at 0. Smoothing both sides alters the optimal discriminator in a way that can let bad samples linger off the data manifold. Smoothing only positives prevents overconfident discriminators without dulling the signal that pushes fakes toward real data.
  • Virtual batch normalization (VBN): Standard batch norm makes an image’s output depend on who else is in the batch. VBN normalizes each example using statistics from a fixed “reference” batch chosen once at the start (plus the example itself). It’s costlier, so they use it in the generator, where that stability matters most.
  • Historical averaging: Add a small penalty that pulls parameters toward their running average over time. Inspired by fictitious play in game theory, it damps cycles and helps when other stabilizers (like labels) are absent.

From pretty pictures to better classifiers

A neat twist turns GANs into semi‑supervised learners. Take a standard K‑class classifier and add one more output: “generated.” Train with two losses:

  • supervised cross‑entropy on labeled data (over the K real classes), and
  • an unsupervised GAN loss that pushes real images to be “not generated” and G(z) to be “generated.”

This “K+1” setup ties the discriminator’s real/fake decision to the classifier’s internal representation. The interaction is delicate: empirically, training the generator with feature matching is what unlocks strong semi‑supervised performance.

Results at the time were state‑of‑the‑art. On MNIST with only 100 labels, a single model made just 93 test errors (beating previous bests); with 2000 labels on SVHN, error dropped to 6.16% (5.88% with an ensemble). On CIFAR‑10 with 4000 labels, an ensemble reached 15.6% error, improving on strong baselines.

Measuring “realism” without a human

Human Turing tests are slow and fickle. The authors popularized an automatic proxy: the Inception score. Run a pretrained ImageNet classifier on generated images. Good images make confident predictions (low entropy p(y|x)) and cover many classes (high entropy p(y) across the set). Combine both as exp(E_x KL(p(y|x) || p(y))). It tracks human judgments well—keeping only the top 1% of samples by this score made humans’ real‑vs‑fake accuracy on CIFAR‑10 drop from 78.7% to 71.4%. The caution: optimize for this score directly, and you’ll produce adversarial cheats, not better images.

What changed on ImageNet—and what didn’t

Scaling to 128×128‑pixel ImageNet without conditioning, a plain DCGAN learned textures and colors but not objects. With these techniques, the model learned fur, eyes, and noses—recognizable parts—yet assembled them with wrong anatomy. It’s progress at unprecedented scale (both resolution and 1000 classes), but also a reminder: global structure is hard.

Why this matters

By turning “win the game” into “match what matters” and giving the discriminator a group sense, these simple tweaks made GANs steadier, more diverse, and more useful. They fooled people on handwritten digits, lifted classifiers with few labels, and gave the field a quick, if imperfect, yardstick. In a game that once just spun in circles, the training finally started to settle—and that’s what made the pictures look real.