Core Components

The core module contains the fundamental building blocks of FIT.

Tensor

Core tensor implementation with automatic differentiation support.

class fit.core.tensor.Tensor(data, requires_grad: bool = False)

Bases: object

Core tensor class with automatic differentiation capabilities.

A tensor stores data and tracks operations for gradient computation.

__init__(data, requires_grad: bool = False)

Initialize a tensor.

Parameters:

• data – The data array (numpy array or compatible)

• requires_grad – Whether to track operations for gradient computation

__repr__() → str

String representation of the tensor.

__add__(other)

Add a tensor or scalar to this tensor.

__radd__(other)

Handle right addition (scalar + tensor).

__mul__(other)

Multiply tensor by another tensor or scalar.

__rmul__(other)

Handle right multiplication (scalar * tensor).

__sub__(other)

Subtract another tensor or scalar from this tensor.

__rsub__(other)

Handle right subtraction (scalar - tensor).

__truediv__(other)

Divide tensor by another tensor or scalar.

__rtruediv__(other)

Handle right division (scalar / tensor).

__pow__(exponent)

Raise tensor to a power.

__matmul__(other)

Matrix multiplication.

sum(axis=None, keepdims=False)

Sum tensor elements along specified axis.

Parameters:

• axis – Axis along which to sum

• keepdims – Whether to keep the summed dimensions

Returns:

A new tensor with summed values

mean(axis=None, keepdims=False)

Calculate the mean of tensor elements along specified axis.

exp()

Calculate the exponential of all tensor elements.

log()

Calculate the natural logarithm of all tensor elements.

backward()

Perform backpropagation starting from this tensor.

__hash__()

Hash function for tensor objects.

__eq__(other)

Check if two tensor objects are the same.

property shape

Return the shape of the tensor data.

property ndim

Return the number of dimensions.

property size

Return the total number of elements.

reshape(*shape)

Reshape the tensor to the given shape.

__getitem__(idx)

Get items from the tensor at specified indices.

property T

Return the transpose of the tensor.

Autograd

This module implements the autograd engine for automatic differentiation.

The autograd engine tracks computations and builds a directed acyclic graph for efficient backpropagation.

class fit.core.autograd.Node(requires_grad: bool = False)

Bases: object

Represents a node in the computational graph.

Each node represents a value in the computation graph and tracks its dependencies (parents) as well as the backward function to compute gradients with respect to its inputs.

__init__(requires_grad: bool = False)

Initialize a node in the computational graph.

Parameters:

requires_grad – Whether this node requires gradient computation

backward(gradient: ndarray | None = None) → None

Perform backpropagation starting from this node.

Parameters:

gradient – Upstream gradient to apply (defaults to ones)

class fit.core.autograd.Function

Bases: object

Base class for autograd functions.

Each function represents an operation in the computation and defines how to compute the forward pass and the backward pass (gradient computation).

static apply(ctx: Dict[str, Any], *inputs: Any) → Any

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray | None, ...]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

classmethod forward(*inputs: Tensor) → Tensor

class fit.core.autograd.Add

Bases: Function

Addition function.

static apply(ctx: Dict[str, Any], a: ndarray, b: ndarray) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray, ndarray]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.Multiply

Bases: Function

Element-wise multiplication function.

static apply(ctx: Dict[str, Any], a: ndarray, b: ndarray) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray, ndarray]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.MatMul

Bases: Function

Matrix multiplication function.

static apply(ctx: Dict[str, Any], a: ndarray, b: ndarray) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray, ndarray]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.Sum

Bases: Function

Sum reduction function.

static apply(ctx: Dict[str, Any], a: ndarray, axis: int | None = None, keepdims: bool = False) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray, None, None]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.Mean

Bases: Function

Mean reduction function.

static apply(ctx: Dict[str, Any], a: ndarray, axis=None, keepdims=False) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray, None, None]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.Exp

Bases: Function

Exponential function.

static apply(ctx: Dict[str, Any], a: ndarray) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.Log

Bases: Function

Natural logarithm function.

static apply(ctx: Dict[str, Any], a: ndarray) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.Reshape

Bases: Function

Reshape function.

static apply(ctx: Dict[str, Any], a: ndarray, shape: Tuple[int, ...]) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray, None]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

class fit.core.autograd.ReLU

Bases: Function

Rectified Linear Unit activation function.

static apply(ctx: Dict[str, Any], a: ndarray) → ndarray

Apply the function to inputs.

Parameters:

• ctx – Context dictionary to store data for the backward pass

• *inputs – Input values

Returns:

Output value(s)

static backward(ctx: Dict[str, Any], grad_output: ndarray) → Tuple[ndarray]

Compute gradients with respect to inputs.

Parameters:

• ctx – Context dictionary with data stored during the forward pass

• grad_output – Gradient with respect to the output

Returns:

Tuple of gradients with respect to inputs

fit.core.autograd.get_function(name: str) → Function

Get a function by name from the registry.

Parameters:

name – Name of the function

Returns:

Function class

Raises:

ValueError – If the function is not found

fit.core.autograd.register_function(name: str, function: Function) → None

Register a new function in the registry.

Parameters:

• name – Name of the function

• function – Function class to register

Operations

This module implements specialized operations for tensors.

These operations build on the core tensor capabilities and autograd engine to provide higher-level functionality.

fit.core.ops.matmul(a: Tensor, b: Tensor) → Tensor

Perform matrix multiplication: a @ b.

Parameters:

• a – First tensor

• b – Second tensor

Returns:

Result tensor

fit.core.ops.transpose(x: Tensor, axes: Tuple[int, ...] | None = None) → Tensor

Transpose a tensor.

Parameters:

• x – Input tensor

• axes – Permutation of dimensions (default is to reverse dimensions)

Returns:

Transposed tensor

fit.core.ops.einsum(equation: str, *tensors: Tensor) → Tensor

Einstein summation for tensors.

Parameters:

• equation – Summation equation in Einstein notation

• *tensors – Input tensors

Returns:

Result tensor

fit.core.ops.sigmoid(x: Tensor) → Tensor

Apply sigmoid activation: 1 / (1 + exp(-x)).

Parameters:

x – Input tensor

Returns:

Output tensor

fit.core.ops.softmax(x: Tensor, axis: int = -1) → Tensor

Apply softmax activation along specified axis.

Parameters:

• x – Input tensor

• axis – Axis along which to apply softmax

Returns:

Output tensor

fit.core.ops.tanh(x: Tensor) → Tensor

Apply hyperbolic tangent activation: (exp(x) - exp(-x)) / (exp(x) + exp(-x)).

Parameters:

x – Input tensor

Returns:

Output tensor

fit.core.ops.abs(x: Tensor) → Tensor

Compute the absolute value of a tensor.

Parameters:

x – Input tensor

Returns:

Output tensor

fit.core.ops.sqrt(x: Tensor) → Tensor

Compute the square root of a tensor.

Parameters:

x – Input tensor

Returns:

Output tensor

fit.core.ops.sin(x: Tensor) → Tensor

Compute the sine of a tensor (in radians).

Parameters:

x – Input tensor

Returns:

Output tensor

fit.core.ops.cos(x: Tensor) → Tensor

Compute the cosine of a tensor (in radians).

Parameters:

x – Input tensor

Returns:

Output tensor

fit.core.ops.zeros(shape: Tuple[int, ...], requires_grad: bool = False) → Tensor

Create a tensor filled with zeros.

Parameters:

• shape – Shape of the tensor

• requires_grad – Whether the tensor requires gradient computation

Returns:

Tensor filled with zeros

fit.core.ops.ones(shape: Tuple[int, ...], requires_grad: bool = False) → Tensor

Create a tensor filled with ones.

Parameters:

• shape – Shape of the tensor

• requires_grad – Whether the tensor requires gradient computation

Returns:

Tensor filled with ones

fit.core.ops.randn(shape: Tuple[int, ...], requires_grad: bool = False) → Tensor

Create a tensor filled with random values from a standard normal distribution.

Parameters:

• shape – Shape of the tensor

• requires_grad – Whether the tensor requires gradient computation

Returns:

Tensor filled with random values

fit.core.ops.concatenate(tensors: List[Tensor], axis: int = 0) → Tensor

Concatenate tensors along specified axis.

Parameters:

• tensors – List of tensors to concatenate

• axis – Axis along which to concatenate

Returns:

Concatenated tensor

fit.core.ops.std(x: Tensor, axis: int | None = None, keepdims: bool = False) → Tensor

Compute the standard deviation of a tensor.

Parameters:

• x – Input tensor

• axis – Axis along which to compute standard deviation

• keepdims – Whether to keep the reduced dimensions

Returns:

Standard deviation tensor

fit.core.ops.var(x: Tensor, axis: int | None = None, keepdims: bool = False) → Tensor

Compute the variance of a tensor.

Parameters:

• x – Input tensor

• axis – Axis along which to compute variance

• keepdims – Whether to keep the reduced dimensions

Returns:

Variance tensor

fit.core.ops.argmax(x: Tensor, axis: int | None = None) → Tensor

Return the indices of the maximum values along specified axis.

Parameters:

• x – Input tensor

• axis – Axis along which to find maximum values

Returns:

Indices tensor

fit.core.ops.logsumexp(x: Tensor, axis: int | None = None, keepdims: bool = False) → Tensor

Compute the log of the sum of exponentials of input elements.

This is a numerically stable version of log(sum(exp(x))).

Parameters:

• x – Input tensor

• axis – Axis along which to perform the operation

• keepdims – Whether to keep the reduced dimensions

Returns:

Result tensor

fit.core.ops.mse_loss(predictions: Tensor, targets: Tensor, reduction: str = 'mean') → Tensor

Compute Mean Squared Error loss.

Parameters:

• predictions – Predicted values

• targets – Target values

• reduction – Reduction method (‘mean’ or ‘sum’)

Returns:

Loss tensor

fit.core.ops.binary_cross_entropy(predictions: Tensor, targets: Tensor, reduction: str = 'mean') → Tensor

Compute Binary Cross Entropy loss.

Parameters:

• predictions – Predicted probabilities (between 0 and 1)

• targets – Binary target values (0 or 1)

• reduction – Reduction method (‘mean’ or ‘sum’)

Returns:

Loss tensor

fit.core.ops.softmax_cross_entropy(logits: Tensor, targets: Tensor, reduction: str = 'mean') → Tensor

Compute Softmax Cross Entropy loss.

Parameters:

• logits – Unnormalized predictions (logits)

• targets – Target class indices or one-hot encoded targets

• reduction – Reduction method (‘mean’ or ‘sum’)

Returns:

Loss tensor

fit.core.ops.cosine_similarity(x1: Tensor, x2: Tensor, dim: int = 1, eps: float = 1e-08) → Tensor

Compute cosine similarity between tensors along specified dimension.

Parameters:

• x1 – First tensor

• x2 – Second tensor

• dim – Dimension along which to compute similarity

• eps – Small value to avoid division by zero

Returns:

Similarity tensor

fit.core.ops.pairwise_distance(x1: Tensor, x2: Tensor, p: float = 2.0, eps: float = 1e-06) → Tensor

Compute pairwise distance between tensors.

Parameters:

• x1 – First tensor (batch_size x D)

• x2 – Second tensor (batch_size x D)

• p – p-norm to use for distance calculation

• eps – Small value to avoid division by zero

Returns:

Distance tensor (batch_size)

Exceptions