Binary Cross-Entropy Loss - Supervised Learning

Binary Cross-Entropy Loss

Implement the mean binary cross-entropy (log loss) between 0/1 labels and predicted probabilities.

Background

BCE is the negative log-likelihood of Bernoulli labels, so minimising it is maximum-likelihood training of a binary classifier (logistic regression, the output head of a binary neural net). It punishes confident mistakes hard: predicting 0.01 for a true positive costs -log(0.01) ≈ 4.6, while 0.9 costs only 0.105.

Formula

p_i   = clip(y_pred_i, eps, 1 - eps),   eps = 1e-15
BCE   = -(1/m) * sum_i [ y_i * log(p_i) + (1 - y_i) * log(1 - p_i) ]

• y_i in {0, 1}: the true label of sample i
• y_pred_i in [0, 1]: the predicted probability that y_i = 1
• m: the number of samples

Inputs

• y_true: array of shape (m,) with 0/1 labels (ints or floats)
• y_pred: array of shape (m,) with probabilities in [0, 1]

Expected Return Format

A Python float: the mean loss rounded to 4 decimals.

Edge Cases and Conventions

• Clip first: log(0) is -inf, so clip y_pred to [1e-15, 1 - 1e-15] before taking logs. With clipping, perfect predictions give a loss that rounds to 0.0, and a prediction of exactly the wrong value costs about 34.54 instead of inf.
• Average over samples (mean, not sum).

Constraints

• NumPy only, vectorised.

Follow-up

• When you have logits z rather than probabilities, why is max(z, 0) - z*y + log(1 + exp(-|z|)) preferred over BCE(y, sigmoid(z))?

Input: binary_cross_entropy([1, 0, 1], [0.9, 0.1, 0.8])
Output: 0.1446

Input: binary_cross_entropy([1, 0], [0.5, 0.5])
Output: 0.6931

import numpy as np

def binary_cross_entropy(y_true, y_pred):
    """
    Compute the mean binary cross-entropy loss.

    Args:
        y_true: True labels (0 or 1), shape (m,)
        y_pred: Predicted probabilities in [0, 1], shape (m,)

    Returns:
        loss: Mean BCE as a Python float, rounded to 4 decimals
              (clip y_pred to [1e-15, 1 - 1e-15] before the logs)
    """
    # Your code here
    pass