Struct PerplexityMetric 

pub struct PerplexityMetric { /* private fields */ }

The perplexity metric.

Perplexity is a measure of how well a probability distribution or probability model predicts a sample. It’s commonly used to evaluate language models. A lower perplexity indicates that the model is more confident in its predictions.

Mathematically, perplexity is defined as the exponentiation of the cross-entropy loss: PPL = exp(H(p, q)) = exp(-1/N * Σ log(p(x_i)))

where:

• H(p, q) is the cross-entropy between the true distribution p and predicted distribution q
• N is the number of tokens
• p(x_i) is the predicted probability of the i-th token

Aggregation

Unlike other metrics, perplexity cannot be simply averaged across batches. This implementation correctly accumulates the total negative log-likelihood and total token count across batches, then computes perplexity as exp(total_nll / total_tokens).

Implementations

impl PerplexityMetric

pub fn new() -> PerplexityMetric

Creates the metric.

pub fn with_pad_token(self, index: usize) -> PerplexityMetric

Sets the pad token to exclude from perplexity calculation.

When a pad token is set, predictions for padding tokens are masked out and do not contribute to the perplexity calculation. This is important for variable-length sequences where padding is used.

Trait Implementations

impl Clone for PerplexityMetric

fn clone(&self) -> PerplexityMetric

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for PerplexityMetric

fn default() -> PerplexityMetric

Returns the “default value” for a type.

impl Metric for PerplexityMetric

type Input = PerplexityInput

The input type of the metric.

fn update( &mut self, input: &PerplexityInput, _metadata: &MetricMetadata, ) -> Result<SerializedEntry, TensorReadError>

Update the metric state and returns the current metric entry.

fn compute(&mut self) -> Result<SerializedEntry, TensorReadError>

Compute the final metric value.

fn clear(&mut self)

Clear the metric state.

fn name(&self) -> Arc<String>

The parameterized name of the metric.

fn attributes(&self) -> MetricAttributes

Attributes of the metric.

fn description(&self) -> Option<String>

A short description of the metric.

impl Numeric for PerplexityMetric

fn value(&self) -> Option<NumericEntry>

Returns the numeric value of the metric, or None if this metric has no meaningful value yet (e.g. it only accumulates state per batch and computes its value once all state has been accumulated).

fn running_value(&self) -> Option<NumericEntry>

Returns the current aggregated value of the metric over the global step, or None if this metric has no meaningful value yet (e.g. it only accumulates state per batch and computes its value once all state has been accumulated).

fn final_value(&self) -> NumericEntry

Returns the final, authoritative value of the metric for the epoch. Only meaningful after Metric::compute has been called.