Classifier-Free Guidance - Generative Models

Classifier-Free Guidance (CFG)

What you'll build

The guidance rule that makes text-to-image diffusion follow the prompt, plus a DDPM reverse step that uses the guided noise prediction.

Background

A conditional diffusion model is trained with its condition randomly dropped (replaced by a null token, typically 10–20% of the time), so one network predicts both ε_c = ε_θ(x_t, t, c) and ε_u = ε_θ(x_t, t, ∅). At sampling time Ho & Salimans (2022) push the prediction further in the direction the condition points:

Formula

ε_guided = ε_u + w · (ε_c − ε_u)

Guidance scale w	Effect
0	unconditional prediction
1	plain conditional prediction (no guidance)
> 1	follows the prompt more closely, less diversity (Stable Diffusion's default is 7.5)

This is the "guidance scale" convention used by most libraries. The paper writes ε̃ = (1 + w')·ε_c − w'·ε_u, the same formula with w = 1 + w'.

Guided step: plug ε_guided into the DDPM reverse step from DDPM Reverse Sampling Step (0-indexed t, σ_t² = β_t, no noise at t = 0):

α_t = 1 − β_t,   ᾱ_t = Π_{s ≤ t} α_s
x_{t-1} = ( x_t − β_t / sqrt(1 − ᾱ_t) · ε_guided ) / sqrt(α_t)  +  σ_t · z     (z term only if t > 0)

Inputs

• eps_uncond, eps_cond, x_t, z: float arrays of the same shape
• w: float guidance scale
• t: int in [0, T−1]; betas: float array (T,)

Expected return format

• guided_noise(eps_uncond, eps_cond, w) returns ε_guided (same shape as the inputs)
• cfg_step(x_t, eps_uncond, eps_cond, w, t, betas, z) returns x_{t-1} (same shape as x_t)

Constraints

NumPy only. Reuse guided_noise inside cfg_step.

Follow-up

• CFG doubles the cost of each step (two forward passes). How do batching and guidance distillation reduce it?
• Why do very high guidance scales oversaturate images?

Input: eps_uncond = [0.1, -0.2], eps_cond = [0.3, 0.2], w = 3
Output: [0.7, 1.0]

Input: x_t = [0.5, -1.0, 2.0], eps_uncond = [0.1, 0.0, -0.3], eps_cond = [0.4, -0.2, 0.1], w = 7.5, t = 3, betas = linspace(0.01, 0.1, 5), z = [1.0, -0.5, 0.25]
Output: [0.3322, -0.8824, 1.6156]

import numpy as np

def guided_noise(eps_uncond, eps_cond, w):
    """eps_uncond + w * (eps_cond - eps_uncond)"""
    # Your code here
    pass


def cfg_step(x_t, eps_uncond, eps_cond, w, t, betas, z):
    """
    One DDPM reverse step using the guided noise prediction.

    Args:
        x_t: current sample
        eps_uncond, eps_cond: unconditional / conditional noise predictions
        w: guidance scale
        t: timestep (int, 0-indexed into betas)
        betas: (T,) noise schedule
        z: standard normal noise (ignored at t = 0)

    Returns:
        x_{t-1}
    """
    # Your code here
    pass