Loss Functions

The loss module provides loss functions for training neural networks.

Regression Losses

Regression loss functions.

class fit.loss.regression.MSELoss(reduction='mean')

Bases: object

Mean Squared Error loss for regression.

L(y, ŷ) = (1/n) * Σ(y - ŷ)²

__init__(reduction='mean')

Initialize MSELoss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute mean squared error loss.

Parameters:

• predictions – Predicted values

• targets – Target values

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of MSE loss.

class fit.loss.regression.MAELoss(reduction='mean')

Bases: object

Mean Absolute Error loss for regression.

L(y, ŷ) = (1/n) * Σ|y - ŷ|

__init__(reduction='mean')

Initialize MAELoss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute mean absolute error loss.

Parameters:

• predictions – Predicted values

• targets – Target values

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of MAE loss.

class fit.loss.regression.SmoothL1Loss(beta=1.0, reduction='mean')

Bases: object

Smooth L1 Loss (Huber Loss) for regression.

Less sensitive to outliers than MSE. L(x) = 0.5*x² if |x| < β

β*|x| - 0.5*β² otherwise

__init__(beta=1.0, reduction='mean')

Initialize SmoothL1Loss.

Parameters:

• beta – Threshold for switching between L2 and L1 loss

• reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute smooth L1 loss.

Parameters:

• predictions – Predicted values

• targets – Target values

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of Smooth L1 loss.

class fit.loss.regression.HuberLoss(delta=1.0, reduction='mean')

Bases: object

Huber Loss for regression.

Combines MSE and MAE - quadratic for small errors, linear for large errors.

__init__(delta=1.0, reduction='mean')

Initialize Huber Loss.

Parameters:

• delta – Threshold for switching between quadratic and linear

• reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute Huber loss.

Parameters:

• predictions – Predicted values

• targets – Target values

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of Huber loss.

class fit.loss.regression.LogCoshLoss(reduction='mean')

Bases: object

Logarithm of the hyperbolic cosine loss.

Smooth approximation to MAE that is less sensitive to outliers than MSE.

__init__(reduction='mean')

Initialize LogCosh Loss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute log-cosh loss.

Parameters:

• predictions – Predicted values

• targets – Target values

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of log-cosh loss.

class fit.loss.regression.QuantileLoss(quantile=0.5, reduction='mean')

Bases: object

Quantile Loss for quantile regression.

Allows predicting specific quantiles of the target distribution.

__init__(quantile=0.5, reduction='mean')

Initialize Quantile Loss.

Parameters:

• quantile – Quantile to predict (0 < quantile < 1)

• reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute quantile loss.

Parameters:

• predictions – Predicted quantile values

• targets – Target values

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of quantile loss.

class fit.loss.regression.CosineSimilarityLoss(reduction='mean')

Bases: object

Cosine Similarity Loss for regression.

Measures the cosine of the angle between prediction and target vectors.

__init__(reduction='mean')

Initialize Cosine Similarity Loss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute cosine similarity loss.

Parameters:

• predictions – Predicted vectors

• targets – Target vectors

Returns:

Loss tensor (1 - cosine_similarity)

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of cosine similarity loss.

Classification Losses

Classification loss functions.

class fit.loss.classification.CrossEntropyLoss(reduction='mean')

Bases: object

Cross-entropy loss for multi-class classification.

Combines log-softmax and negative log-likelihood in a numerically stable way.

__init__(reduction='mean')

Initialize CrossEntropyLoss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(logits: Tensor, targets: Tensor) → Tensor

Compute cross-entropy loss.

Parameters:

• logits – Raw model outputs (batch_size, num_classes)

• targets – Target class indices (batch_size,) or one-hot (batch_size, num_classes)

Returns:

Loss tensor

forward(logits: Tensor, targets: Tensor) → Tensor

Forward pass of cross-entropy loss.

class fit.loss.classification.BinaryCrossEntropyLoss(reduction='mean')

Bases: object

Binary cross-entropy loss for binary classification.

__init__(reduction='mean')

Initialize BinaryCrossEntropyLoss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(predictions: Tensor, targets: Tensor) → Tensor

Compute binary cross-entropy loss.

Parameters:

• predictions – Predicted probabilities (batch_size,) or (batch_size, 1)

• targets – Target labels (batch_size,) - should be 0 or 1

Returns:

Loss tensor

forward(predictions: Tensor, targets: Tensor) → Tensor

Forward pass of binary cross-entropy loss.

class fit.loss.classification.NLLLoss(reduction='mean')

Bases: object

Negative Log-Likelihood loss.

Expects log-probabilities as input (e.g., output of log-softmax).

__init__(reduction='mean')

Initialize NLLLoss.

Parameters:

reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(log_probs: Tensor, targets: Tensor) → Tensor

Compute negative log-likelihood loss.

Parameters:

• log_probs – Log probabilities (batch_size, num_classes)

• targets – Target class indices (batch_size,)

Returns:

Loss tensor

forward(log_probs: Tensor, targets: Tensor) → Tensor

Forward pass of NLL loss.

class fit.loss.classification.FocalLoss(alpha=1.0, gamma=2.0, reduction='mean')

Bases: object

Focal Loss for addressing class imbalance.

Focal Loss = -α(1-p)^γ * log(p)

__init__(alpha=1.0, gamma=2.0, reduction='mean')

Initialize Focal Loss.

Parameters:

• alpha – Weighting factor for rare class

• gamma – Focusing parameter

• reduction – Reduction method (‘mean’, ‘sum’, or ‘none’)

__call__(logits: Tensor, targets: Tensor) → Tensor

Compute focal loss.

Parameters:

• logits – Raw model outputs (batch_size, num_classes)

• targets – Target class indices (batch_size,)

Returns:

Loss tensor

forward(logits: Tensor, targets: Tensor) → Tensor

Forward pass of focal loss.

Advanced Losses