Int8 Weight Quantization (absmax) - Post-training & Efficiency

Int8 Weight Quantization (absmax)

What you'll build

Symmetric (absmax) int8 quantization of a weight matrix, per output channel and per tensor; dequantization; an int8 matmul that rescales an int32 accumulator; and a report comparing the two schemes' errors.

Background

Storing weights as int8 instead of float32 cuts memory by 4×, and int8 matmuls are fast on modern hardware. Absmax quantization maps the largest magnitude to ±127 and rounds everything else to the nearest step. The catch is outliers: one large weight sets a large step for everything sharing its scale. Giving each output row its own scale (per-channel, the "vector-wise" quantization of LLM.int8()) contains that damage.

Formula

For a group of weights w that share one scale:

scale = max(max|w|, 1e-8) / 127
w_q   = clip(round(w / scale), -127, 127)      stored as int8
w_hat = w_q · scale                            dequantized

Rounding is round-half-to-even (np.round / torch.round), so the per-weight error is at most scale / 2.

• Per-tensor: one scale for the whole matrix.
• Per-channel: one scale per row of W, which has shape (d_out, d_in).

Int8 matmul for y = x · Wᵀ:

1. Quantize the activations per row of x (one scale per token): x_q of shape (n, d_in) and s_x of shape (n,).
2. Accumulate the integer products in int32: acc = x_q · W_qᵀ, shape (n, d_out).
3. Rescale with both scales: y ≈ acc · s_x[:, None] · s_w[None, :].

Inputs

• W: (d_out, d_in) float array
• x: (n, d_in) float array
• W_q: (d_out, d_in) int8 array; w_scales / scales: (d_out,) float array (or a Python float in dequantize)

Expected return format

• quantize_per_channel(W) returns a tuple (W_q, scales): an int8 array of shape (d_out, d_in) and a float array of shape (d_out,)
• quantize_per_tensor(W) returns (W_q, scale): an int8 array and a Python float
• dequantize(W_q, scales) returns a float array of W_q's shape; scales is a (d_out,) array (one per row) or a scalar
• int8_matmul(x, W_q, w_scales) returns an (n, d_out) float array approximating x · Wᵀ
• quantization_report(W) returns a dictionary:
  • 'per_channel_error': max |W - dequantize(*quantize_per_channel(W))| as a Python float
  • 'per_tensor_error': max |W - dequantize(*quantize_per_tensor(W))| as a Python float

Edge cases and conventions

• An all-zero row must not divide by zero: the 1e-8 floor on the absmax keeps it at 0.
• The symmetric range is [-127, 127]; -128 is never used.
• Exact halves round to even, e.g. 63.5 becomes 64 and 62.5 becomes 62.

Constraints

NumPy only, vectorized. Cast to int32 before the integer matmul: int8 products overflow int8 immediately.

Follow-up

• Why do LLM activations need special care (outlier features), and how does LLM.int8() handle them with mixed precision?
• How do zero-point (asymmetric) quantization and 4-bit group-wise quantization (e.g. block size 64) change the error?

Input: quantize_per_channel(np.array([[0.7, -1.2, 0.3], [2.0, 0.1, -0.5]]))
Output: (array([[74, -127, 32], [127, 6, -32]], dtype=int8), array([0.009449, 0.015748]))

Input: quantization_report(np.array([[0.01, -0.02, 0.013], [5.0, -3.0, 1.7]]))
Output: {'per_channel_error': 0.007874, 'per_tensor_error': 0.01937}

import numpy as np

def quantize_per_channel(W):
    """
    Absmax-quantize each row of W with its own scale.

    Args:
        W: (d_out, d_in) float weights

    Returns:
        (W_q, scales): W_q int8 (d_out, d_in), scales float (d_out,)
        with scales = max(row absmax, 1e-8) / 127
    """
    # Your code here
    pass


def quantize_per_tensor(W):
    """
    Absmax-quantize all of W with one scale.

    Returns:
        (W_q, scale): W_q int8 with W's shape, scale a Python float
    """
    # Your code here
    pass


def dequantize(W_q, scales):
    """
    Map int8 values back to floats.

    Args:
        W_q: (d_out, d_in) int8 array
        scales: (d_out,) per-row scales, or a single scalar

    Returns:
        (d_out, d_in) float array W_q * scale
    """
    # Your code here
    pass


def int8_matmul(x, W_q, w_scales):
    """
    Approximate x @ W.T with int8 operands.

    Args:
        x: (n, d_in) float activations (quantize per row)
        W_q: (d_out, d_in) int8 weights
        w_scales: (d_out,) per-row weight scales

    Returns:
        (n, d_out) float array
    """
    # Your code here
    pass


def quantization_report(W):
    """Returns {'per_channel_error': float, 'per_tensor_error': float} (max abs errors)."""
    # Your code here
    pass