burn::nn::loss

Struct SmoothL1Loss

pub struct SmoothL1Loss {
    pub beta: f32,
}

Expand description

Computes the Smooth L1 Loss between predictions and targets.

This loss function uses L2 loss for small errors (below beta) and L1 loss for large errors (above beta), providing robustness to outliers while maintaining smooth gradients near |x - y| = 0.

§Mathematical Definition

For predictions x and targets y, the element-wise loss is:

L_i = 0.5 * (x_i - y_i)² / beta , if |x_i - y_i| < beta
L_i = |x_i - y_i| - 0.5 * beta , otherwise

§Notes

Smooth L1 loss is closely related to HuberLoss since it is equivalent to HuberLoss scaled by 1/beta: SmoothL1(x, y, beta) = Huber(x, y, beta) / beta

This leads to the following differences:

As beta approaches 0, Smooth L1 loss converges to L1Loss, while HuberLoss converges to 0. When beta = 0, Smooth L1 loss is equivalent to L1 loss. Thus, the beta parameter in Burn must be positive. L1Loss should be used for beta = 0.
As beta approaches positive infinity, Smooth L1 loss converges to a constant 0 loss, while HuberLoss converges to L2Loss.

§Example

use burn_nn::loss::{SmoothL1LossConfig, Reduction};
use burn::tensor::Tensor;

// Create Smooth L1 loss with the default beta=1.0
let smooth_l1 = SmoothL1LossConfig::new().init();

let predictions: Tensor<Backend, 2> = /* model output */;
let targets: Tensor<Backend, 2> = /* ground truth */;

// Compute element-wise loss without reduction
let element_wise = smooth_l1.forward(predictions.clone(), targets.clone());

// Compute loss with mean reduction
let loss = smooth_l1.forward_with_reduction(predictions.clone(), targets.clone(), Reduction::Mean);

// Per-image loss: reduce over C, H, W → [batch, 1, 1, 1]
let loss_per_image = smooth_l1.forward_reduce_dims(predictions, targets, &[1, 2, 3]);

Fields§

§beta: f32

Specifies the threshold at which to change between L1 and L2 loss. The value must be positive. Default: 1.0

Implementations§

§

impl SmoothL1Loss

pub fn forward<const D: usize, B>( &self, predictions: Tensor<B, D>, targets: Tensor<B, D>, ) -> Tensor<B, D>
where B: Backend,

Computes the element-wise smooth L1 loss without reduction.

§Arguments

predictions - The model’s predicted values.
targets - The ground truth target values.

§Returns

A tensor of the same shape as the inputs, containing the smooth L1 loss for each element.

§Shapes

predictions: [...dims] - Any shape
targets: [...dims] - Must match predictions shape
output: [...dims] - Same shape as inputs

pub fn forward_with_reduction<const D: usize, B>( &self, predictions: Tensor<B, D>, targets: Tensor<B, D>, reduction: Reduction, ) -> Tensor<B, 1>
where B: Backend,

Computes the smooth L1 loss with reduction.

§Arguments

predictions - The model’s predicted values.
targets - The ground truth target values.
reduction - Specifies how to reduce the element-wise losses:
- Reduction::Mean or Reduction::Auto: Returns the mean of all element-wise losses.
- Reduction::Sum: Returns the sum of all element-wise losses.

§Returns

A scalar tensor containing the reduced loss value.

§Shapes

predictions: [...dims] - Any shape
targets: [...dims] - Must match predictions shape
output: [1] - Scalar loss value

pub fn forward_reduce_dims<const D: usize, B>( &self, predictions: Tensor<B, D>, targets: Tensor<B, D>, dims: &[usize], ) -> Tensor<B, D>
where B: Backend,

Computes the smooth L1 loss with reduction over specified dimensions.

Calculates element-wise smooth L1 loss, then takes the mean over the specified dimensions. Useful for per-sample or per-channel losses.

Dimensions can be provided in any order. They are sorted internally and reduced from highest to lowest to ensure indices remain valid.

§Arguments

predictions - The model’s predicted values.
targets - The ground truth target values.
dims - Dimensions to reduce over.

§Returns

A tensor with the specified dimensions reduced to size 1.

§Example

// Consider image tensor with shape [batch, C, H, W]
let smooth_l1 = SmoothL1LossConfig::new().init();

// Per-image loss: reduce over C, H, W → [batch, 1, 1, 1]
let loss_per_image = smooth_l1.forward_reduce_dims(predictions, targets, &[1, 2, 3]);