Tensor Operations - PyTorch Basics

Tensor Operations

What you'll build: tensor_ops(x, y), which applies the everyday arithmetic, reduction and shape operations to two (2, 3) tensors and returns the results in a dictionary.

Background

Most model code is a handful of operations: element-wise arithmetic, reductions over a dimension (sum, mean, max) and reshapes. PyTorch and NumPy share almost all of it; the differences that bite are the keyword names and defaults:

Concept	PyTorch	NumPy
Reduce over a dimension	x.sum(dim=0)	x.sum(axis=0)
Keep the reduced dimension	keepdim=True	keepdims=True
Standard deviation	x.std() is unbiased (divides by n-1)	x.std() divides by n
Add a size-1 dimension	x.unsqueeze(0)	np.expand_dims(x, 0) or x[None]
Reshape	x.view(3, 2) / x.reshape(3, 2)	x.reshape(3, 2)

Inputs

• x, y: tensors of shape (2, 3). y has no zeros.

Expected Return Format

Return a dictionary with these keys:

• Arithmetic (shape (2, 3)): 'add' = x + y, 'sub' = x - y, 'mul' = x * y (element-wise), 'div' = x / (y + 1e-8), 'pow' = x ** 2
• Reductions:
  • 'sum_all': sum of every element (scalar)
  • 'sum_axis0': sum over dim 0 (down the rows), shape (3,)
  • 'sum_axis1': sum over dim 1 (along each row), shape (2,)
  • 'mean': mean of every element (scalar)
  • 'std': population standard deviation of every element (divide by n, not n - 1)
  • 'max': largest element (scalar)
  • 'argmax': index of the largest element in the flattened tensor (an integer from 0 to 5)
• Shape ops:
  • 'reshape': x as shape (3, 2), keeping row-major order
  • 'flatten': x as shape (6,)
  • 'unsqueeze': x with a new leading dimension, shape (1, 2, 3)
  • 'squeeze': the unsqueezed tensor with that dimension removed again, shape (2, 3)
  • 'transpose': x transposed, shape (3, 2)

Edge Cases and Conventions

• The 1e-8 in 'div' guards against division by zero; with the tests' inputs it changes nothing visible.
• 'std' must use the population formula: sqrt(mean((x - mean(x)) ** 2)). In PyTorch pass unbiased=False (or correction=0).

Constraints

Use built-in tensor operations; no Python loops.

Follow-up

• view fails on a non-contiguous tensor (for example after .T) while reshape copies when it must. Why?
• What does x.sum(dim=0, keepdim=True) return, and why is it handy for broadcasting?

Input: r = tensor_ops(np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]), np.array([[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]))
r["add"], r["sum_axis0"], r["sum_axis1"]
Output: (array([[2., 3., 4.], [6., 7., 8.]]), array([5., 7., 9.]), array([ 6., 15.]))

Input: round(float(r["std"]), 4), int(r["argmax"]), r["reshape"].tolist()
Output: (1.7078, 5, [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])

import numpy as np

def tensor_ops(x: np.ndarray, y: np.ndarray) -> dict:
    """
    Perform common tensor operations.

    Args:
        x: First tensor (2, 3)
        y: Second tensor (2, 3), no zeros

    Returns:
        dict with arithmetic keys 'add', 'sub', 'mul', 'div', 'pow';
        reductions 'sum_all', 'sum_axis0', 'sum_axis1', 'mean', 'std'
        (population), 'max', 'argmax' (flat index); and shape ops
        'reshape' (3, 2), 'flatten' (6,), 'unsqueeze' (1, 2, 3),
        'squeeze' (2, 3), 'transpose' (3, 2)
    """
    # Your code here
    pass