Conditional generation: generating from a hint

The condition picks which examples the model draws from

A plain generative model draws from the whole collection. A conditional one draws only from the examples that match a condition c. We write that distribution p(x | c), read "the distribution of x given c". The model turns noise into an example, as before, but the condition changes which function it applies.

Your turn. What does each condition give for the noise draw z = 2?

Same condition, new noise: a new example that still fits

In two dimensions, let the examples be points of two kinds: points on a circle and points along a line. The condition names the kind.

Without a condition, the model draws from the whole collection. That is the same as first picking a condition at random, as often as it occurs in the collection, and then drawing an example for it. In symbols, p(x) is the mix of the p(x | c), each weighted by how common c is.

Training uses examples paired with their conditions

Each training example comes with its condition: a photo with its caption, a height with its age group, a complete shape with the part of it the model will be shown. The model receives the condition as extra input numbers, next to the noise. A label becomes a vector through an embedding. A sentence or a set of points can be read with cross-attention. Sometimes the condition is cut from the example itself, as in shape completion.

A conditional model gives many answers where a predictor gives one

A predictor (a regression model) returns one answer for each input. Trained with squared error, it returns the average of the answers it saw for that input (see loss). For the answers −1 and +1, predicting 0 gives squared errors 1 + 1 = 2, and predicting +1 gives 4 + 0 = 4. That works when one answer is right. It fails when several different answers are right, because their average may be none of them.

conditional_bell.py

import numpy as np

rng = np.random.default_rng(1)

# One model, two conditions: the condition picks the mean and the spread.
params = {"child": (130.0, 8.0), "adult": (170.0, 7.0)}

def generate(condition, n):
    mu, sigma = params[condition]
    return mu + sigma * rng.standard_normal(n)

print("child:", np.round(generate("child", 4), 1))
print("adult:", np.round(generate("adult", 4), 1))

# A fork in a path: it ends at -1 (left) or +1 (right), half the time each.
ends = rng.choice([-1.0, 1.0], size=1000)
print("single best guess (the average):", round(ends.mean(), 2))
print("generated ends:", ends[:6])

child: [132.8 136.6 132.6 119.6]
adult: [176.3 173.1 166.2 174.1]
single best guess (the average): -0.02
generated ends: [ 1.  1. -1. -1.  1.  1.]