Hinge Loss - Supervised Learning

Hinge Loss

Implement the mean hinge loss, the loss behind Support Vector Machines (SVMs).

Background

A linear SVM scores a sample with s = x · w + b and predicts sign(s). The hinge loss asks for more than the right sign: it wants a margin y * s >= 1. Samples beyond the margin cost nothing (that is why only the support vectors near the boundary shape the solution); samples inside the margin or on the wrong side pay linearly. You will minimise this loss in Linear SVM with Subgradient Descent.

Formula

L = (1/n) * sum_i max(0, 1 - y_i * s_i)

• y_i in {-1, +1}: the true label (note: not 0/1)
• s_i: the raw score (a real number, not a probability)
• y_i * s_i: the functional margin; positive means correctly classified
• n: the number of samples

Inputs

• y_true: array of shape (n,) with values in {-1, +1}
• scores: float array of shape (n,)

Expected Return Format

A Python float: the mean hinge loss rounded to 4 decimals.

Edge Cases and Conventions

• A margin of exactly 1 costs 0 (max(0, 0) = 0).
• A correct but unconfident prediction (0 < y*s < 1) still has a positive loss.

Constraints

• NumPy only, vectorised.

Follow-up

• Compare the hinge loss with the logistic loss log(1 + exp(-y*s)) as functions of the margin. Which one ever reaches exactly 0, and what does that imply for sparsity of the solution?

Input: hinge_loss([1, -1, 1], [0.5, -0.8, 1.5])
Output: 0.2333

Input: hinge_loss([1, -1], [2.0, -3.0])
Output: 0.0

import numpy as np

def hinge_loss(y_true, scores):
    """
    Compute the mean hinge loss.

    Args:
        y_true: True labels in {-1, +1}, shape (n,)
        scores: Raw model scores, shape (n,)

    Returns:
        Mean hinge loss as a Python float, rounded to 4 decimals
    """
    # Your code here
    pass