Sigmoid for Logistic Regression - Supervised Learning

Sigmoid for Logistic Regression

Implement the sigmoid (logistic) function, the piece that turns a linear score into a probability in logistic regression.

Background

Logistic regression computes a real-valued score z = x · w + b (the logit) and squashes it into (0, 1) with the sigmoid, so the output can be read as P(y = 1 | x). The sigmoid is also the inverse of the log-odds: if p = sigmoid(z) then z = log(p / (1 - p)). You will reuse it in Logistic Regression with Gradient Descent later in this section.

Formula

sigmoid(z) = 1 / (1 + exp(-z))

Useful properties:

• sigmoid(0) = 0.5, and the output is strictly between 0 and 1 for finite z
• Symmetry: sigmoid(-z) = 1 - sigmoid(z)
• Derivative: sigmoid'(z) = sigmoid(z) * (1 - sigmoid(z)) (at most 0.25, at z = 0)

Inputs

• x: a NumPy array of any shape (ints or floats)

Expected Return Format

A float NumPy array with the same shape as x, with the sigmoid applied element-wise.

Edge Cases and Conventions

• For very negative inputs, exp(-z) overflows to inf (NumPy warns, and the result is still 0). A numerically stable version uses 1 / (1 + exp(-z)) for z >= 0 and the equivalent exp(z) / (1 + exp(z)) for z < 0, so exp only ever sees non-positive numbers.
• Tests compare with a tolerance of about 1e-5.

Constraints

• NumPy only, vectorised (no Python loop over elements).

Follow-up

• Why is sigmoid(z) used for binary classification while softmax is used for K > 2 classes? Show that a 2-class softmax reduces to a sigmoid of the logit difference.

Input: sigmoid(np.array([0, 1, -1]))
Output: array([0.5, 0.731059, 0.268941])

Input: sigmoid(np.array([[2.0, -3.0]]))
Output: array([[0.880797, 0.047426]])

import numpy as np

def sigmoid(x: np.ndarray) -> np.ndarray:
    """
    Compute the sigmoid activation function element-wise.

    Args:
        x: Input array of any shape

    Returns:
        Float array of the same shape, with 1 / (1 + exp(-x)) applied element-wise
    """
    # Your code here
    pass