Focal Loss - CNNs & Computer Vision

Focal Loss

What you'll build

The binary focal loss of RetinaNet (Lin et al., 2017), averaged over all examples.

Background

A one-stage detector scores ~100k anchors per image, and almost all of them are easy background. With plain cross-entropy, those easy negatives each contribute a small loss, but together they swamp the few hard, informative examples. Focal loss multiplies cross-entropy by (1 - p_t)^gamma, which shrinks the loss of examples the model already gets right, and adds a class weight alpha.

Formula

For each example with predicted probability p (of class 1) and label y in {0, 1}:

p_t     = p          if y == 1,   1 - p      if y == 0      # probability of the true class
alpha_t = alpha      if y == 1,   1 - alpha  if y == 0      # alpha weights the POSITIVE class
FL      = -alpha_t * (1 - p_t)^gamma * log(p_t)
loss    = mean(FL) over all examples

• gamma >= 0 is the focusing parameter (default 2.0). gamma = 0 gives alpha-weighted binary cross-entropy.
• alpha in [0, 1] is the positive-class weight (default 0.25, so positives get 0.25 and negatives 0.75, as in the paper).

Example	p_t	(1 - p_t)^2	Effect
easy, p_t = 0.9	0.9	0.01	loss scaled down 100x
hard, p_t = 0.1	0.1	0.81	loss almost unchanged

Inputs

• predictions: 1D array or list of probabilities in [0, 1] (already passed through a sigmoid, not logits).
• targets: 1D array or list of the same length with labels 0 or 1.
• alpha: float, default 0.25; gamma: float, default 2.0.

Expected return format

A Python float: the mean focal loss, rounded to 4 decimal places.

Edge cases and conventions

• Clip p to [1e-7, 1 - 1e-7] before the log, so p = 0 or p = 1 gives a large but finite loss instead of inf or nan.
• The mean is over all examples. (RetinaNet itself sums and divides by the number of positive anchors; that is not asked here.)

Constraints

NumPy only, vectorised (no Python loop over examples).

Follow-up

• Why does the paper also initialise the final classification bias to -log((1 - pi) / pi) with pi = 0.01?
• Rewrite the loss in terms of logits so it is numerically stable for large |logit|.

Input: predictions = [0.9, 0.1], targets = [1, 0], alpha = 0.25, gamma = 2.0
Output: 0.0005

Input: predictions = [0.1, 0.9], targets = [1, 0], alpha = 0.25, gamma = 2.0
Output: 0.9325

import numpy as np

def focal_loss(predictions, targets, alpha=0.25, gamma=2.0):
    """
    Mean binary focal loss.

    Args:
        predictions: Probabilities of class 1, shape (n,), values in [0, 1]
        targets: Labels 0 or 1, shape (n,)
        alpha: Weight of the positive class (negatives get 1 - alpha), default 0.25
        gamma: Focusing parameter, default 2.0

    Returns:
        Mean focal loss as a Python float, rounded to 4 decimal places
        (clip p to [1e-7, 1 - 1e-7] before the log)
    """
    # Your code here
    pass