Diffusion Noise Schedule - Generative Models

Diffusion Noise Schedule

What you'll build

The linear β schedule from DDPM (Ho et al., 2020) and the two quantities every diffusion step uses: α_t and the cumulative product ᾱ_t.

Background

The forward process adds a little Gaussian noise at each of T steps; β_t is the variance added at step t. Because Gaussians compose, you never simulate the steps one by one: the noise level after t steps is fully described by ᾱ_t, which falls from about 1 (clean data) towards 0 (pure noise).

Formula

Timesteps are 0-indexed, t = 0, 1, ..., T−1 (the same indexing as the sampling problems):

β_t  = β_start + t · (β_end − β_start) / (T − 1)     # evenly spaced, both ends included
α_t  = 1 − β_t
ᾱ_t  = α_0 · α_1 · ... · α_t                         # cumulative product, ᾱ_0 = α_0

With the DDPM defaults (T = 1000, β from 1e-4 to 0.02), ᾱ is 0.9999 at t = 0, about 0.0786 at t = 499 and 4.0e-5 at t = 999.

Inputs

• T: int ≥ 2, the number of steps
• beta_start, beta_end: floats with 0 < beta_start < beta_end < 1

Expected return format

A tuple of three float arrays, each of shape (T,): (betas, alphas, alpha_bars)

Constraints

NumPy only, no loops.

Follow-up

• Why did Improved DDPM switch to a cosine schedule? (The linear one destroys information too early for small images.)
• What is the signal-to-noise ratio ᾱ_t / (1 − ᾱ_t) at the last step, and why should it be close to 0?

Input: T = 3, beta_start = 0.1, beta_end = 0.3
Output: betas = [0.1, 0.2, 0.3], alphas = [0.9, 0.8, 0.7], alpha_bars = [0.9, 0.72, 0.504]

Input: T = 1000 (DDPM defaults)
Output: alpha_bars[0] = 0.9999, alpha_bars[-1] ≈ 4.0e-05

import numpy as np

def linear_noise_schedule(T, beta_start=0.0001, beta_end=0.02):
    """
    Create the linear DDPM noise schedule (0-indexed timesteps).

    Args:
        T: Number of diffusion steps (int)
        beta_start: beta_0
        beta_end: beta_{T-1}

    Returns:
        (betas, alphas, alpha_bars): float arrays of shape (T,), where
        alphas = 1 - betas and alpha_bars is the cumulative product of alphas
    """
    # Your code here
    pass