Classifier-free guidance

Two predictions, then a step past the conditional one

Write v(x, t, c) for the velocity the network predicts with the condition c, and v(x, t, ∅) for its prediction with no condition. The symbol ∅ stands for a special "no condition" input. The guided velocity is

v_guided = v(x, t, ∅) + s × ( v(x, t, c) − v(x, t, ∅) ).

• s = 0 gives the unconditional prediction.
• s = 1 gives the conditional prediction.
• s between 0 and 1 lands between the two.
• s above 1 extrapolates: it goes past the conditional prediction, further in the direction that leads from "no condition" to "condition".

Generation then follows v_guided with Euler steps, exactly as it would follow the plain prediction.

Your turn. With the same two predictions, what are the guided velocities for s = 2 and s = 0.5?

Training hides the condition now and then, so one network learns both

Guidance needs both predictions from the same network. Training gets them with condition dropout:

1. For each training example, draw a random number.
2. With a small probability, often 10%, replace the example's condition with the "no condition" input ∅.
3. Train exactly as usual on the result.

The network then learns the conditional velocity from examples that keep their condition. It learns the unconditional velocity from examples whose condition was hidden. Nothing else in training changes. At generation time, each step runs the network twice, once with c and once with ∅, often as one batch of two.

The name comes from history. An earlier method, classifier guidance, steered a diffusion model with a separate classifier trained on noisy examples. Classifier-free guidance gets the same kind of steering from the generative network alone, with no classifier.

On a toy, a larger scale pushes samples away from the other class

Take two classes of 2D points. Class A is a bell curve around (−1, 0), class B one around (1, 0), each with spread 0.5. Without a condition, the data mix both classes equally. For this toy, the exact flow-matching velocities can be computed, so they stand in for a trained network. The condition is "class A".

The program below runs the same toy with 4,000 samples and 100 Euler steps.

guidance_toy.py

import numpy as np

rng = np.random.default_rng(0)
CENTERS = {"A": np.array([-1.0, 0.0]), "B": np.array([1.0, 0.0])}
SPREAD = 0.5                                  # each class: a bell curve with this spread

def velocity(x, t, classes):
    # Exact flow-matching velocity when the data are an equal mix of the listed
    # classes. It stands in for a trained network on this toy.
    s2 = t**2 * SPREAD**2 + (1 - t)**2        # squared spread of x_t within a class
    k = (t * SPREAD**2 - (1 - t)) / s2
    vs = [CENTERS[c] + k * (x - t * CENTERS[c]) for c in classes]
    # np.stack(..., 1) makes one column per class; .sum(1) adds across each row
    logw = np.stack([-((x - t * CENTERS[c])**2).sum(1) / (2 * s2) for c in classes], 1)
    w = np.exp(logw - logw.max(1, keepdims=True))
    w = w / w.sum(1, keepdims=True)           # how likely each class is, given x
    # w[:, [i]] is column i of w, kept as a column so that it lines up with the rows of v
    return sum([w[:, [i]] * v for i, v in enumerate(vs)])

def generate(scale, n=4000, steps=100):
    x = rng.standard_normal((n, 2))           # noise at t = 0
    for k in range(steps):
        t = k / steps
        v_u = velocity(x, t, ["A", "B"])      # without the condition
        v_c = velocity(x, t, ["A"])           # with the condition "class A"
        x = x + (v_u + scale * (v_c - v_u)) / steps
    return x

for scale in [0, 1, 3]:
    x = generate(scale)
    print(f"scale {scale}: mean x {x[:, 0].mean():.2f},",
          f"spread x {x[:, 0].std():.2f}, spread y {x[:, 1].std():.2f}")

scale 0: mean x -0.01, spread x 1.12, spread y 0.49
scale 1: mean x -0.99, spread x 0.49, spread y 0.49
scale 3: mean x -1.53, spread x 0.30, spread y 0.49

Scale 1 reproduces class A: mean −0.99, spread 0.49. Scale 3 pushes the samples past A's center, away from B, and narrows them from left to right. In real models, a scale above 1 often makes samples match their condition more clearly, at the cost of variety. Too large a scale distorts them.

Why it works: the difference points toward "more like the condition"

The difference v(x, t, c) − v(x, t, ∅) is the part of the velocity that the condition adds. Scaling it up exaggerates what makes examples of c differ from examples in general. In the toy, A differs from the mix only from left to right, so guidance acts only from left to right.

In terms of probability, guidance with scale s changes the distribution at each noise level. It starts from the unconditional distribution p(x) and weights each point x by p(c | x)^s. Here p(c | x) is the probability that a classifier of noisy inputs gives to c. At s = 1 the result, p(x) × p(c | x), is proportional to the conditional distribution p(x | c) by Bayes' rule. A larger s favors points that the classifier is surer about.