Smooth L1 Loss - CNNs & Computer Vision

Smooth L1 Loss

What you'll build

The Smooth L1 loss used for bounding-box regression in Fast/Faster R-CNN and SSD, averaged over all elements.

Background

Box regression targets can be far off early in training. L2 (MSE) gives those outliers huge gradients; L1 has constant-size gradients but a kink at 0 that makes fine convergence jittery. Smooth L1 is quadratic near zero and linear beyond beta, so it is smooth at the optimum and robust to outliers. Its gradient magnitude is |x| / beta inside the quadratic zone and capped at 1 outside.

Formula

With x = prediction - target (elementwise) and threshold beta > 0:

L(x) = 0.5 * x^2 / beta      if |x| < beta
       |x| - 0.5 * beta      otherwise
loss = mean of L(x) over all elements

The two pieces meet with equal value (0.5 * beta) and slope (1) at |x| = beta.

Relation to Huber loss: Huber_delta(x) = 0.5 x^2 for |x| < delta, else delta * (|x| - 0.5 delta). So SmoothL1_beta(x) = Huber_beta(x) / beta; the two are identical only when beta = 1. PyTorch has both (F.smooth_l1_loss and F.huber_loss).

Inputs

• predictions, targets: arrays (or lists) of the same shape, any number of dimensions. In detection they are (num_boxes, 4) box offsets.
• beta: float > 0, default 1.0.

Expected return format

A Python float: the mean over every element, rounded to 4 decimal places.

Edge cases and conventions

• |x| == beta falls in the linear branch (use < for the quadratic one); both give 0.5 * beta there anyway.
• Average over all elements, not per box.

Constraints

NumPy only, vectorised with np.where.

Follow-up

• What happens as beta -> 0 and as beta grows large?
• Why did later detectors move to IoU-based box losses (GIoU, DIoU)?

Input: predictions = [0.5, 1.5], targets = [0.0, 0.0], beta = 1.0
Output: 0.5625

Input: predictions = [0.3, 2.0], targets = [0.0, 0.0], beta = 0.5
Output: 0.92

import numpy as np

def smooth_l1_loss(predictions, targets, beta=1.0):
    """
    Mean Smooth L1 loss.

    Args:
        predictions: Predicted values, any shape
        targets: Target values, same shape as predictions
        beta: Quadratic-to-linear transition point, > 0 (default 1.0)

    Returns:
        Mean loss over all elements as a Python float, rounded to 4 decimal places
    """
    # Your code here
    pass