Struct EvaluatorBuilder

pub struct EvaluatorBuilder<B, TI, TO>
where
    B: Backend,
    TO: ItemLazy,
{ /* private fields */ }

Struct to configure and create an evaluator.

The generics components of the builder should probably not be set manually, as they are optimized for Rust type inference.

Implementations

impl<B, TI, TO> EvaluatorBuilder<B, TI, TO>

where B: Backend, TO: ItemLazy + 'static,

pub fn new(directory: impl AsRef<Path>) -> EvaluatorBuilder<B, TI, TO>

Creates a new evaluator builder.

Arguments

• directory - The directory to save the checkpoints.

pub fn metrics<M>(self, metrics: M) -> EvaluatorBuilder<B, TI, TO>

where M: EvalMetricRegistration<TI, TO>,

Registers numeric test metrics.

pub fn metrics_text<M>(self, metrics: M) -> EvaluatorBuilder<B, TI, TO>

where M: EvalTextMetricRegistration<TI, TO>,

Registers text metrics.

pub fn with_application_logger( self, logger: Option<Box<dyn ApplicationLoggerInstaller>>, ) -> EvaluatorBuilder<B, TI, TO>

By default, Rust logs are captured and written into evaluation.log. If disabled, standard Rust log handling will apply.

pub fn metric_numeric<Me>(self, metric: Me) -> EvaluatorBuilder<B, TI, TO>

where Me: Metric + Numeric + 'static, <TO as ItemLazy>::ItemSync: Adaptor<<Me as Metric>::Input>,

Register a numeric test metric.

pub fn metric<Me>(self, metric: Me) -> EvaluatorBuilder<B, TI, TO>

where Me: Metric + 'static, <TO as ItemLazy>::ItemSync: Adaptor<<Me as Metric>::Input>,

Register a text test metric.

pub fn renderer( self, renderer: Box<dyn MetricsRenderer>, ) -> EvaluatorBuilder<B, TI, TO>

Replace the default CLI renderer with a custom one.

Arguments

• renderer - The custom renderer.

pub fn summary(self) -> EvaluatorBuilder<B, TI, TO>

Enable the evaluation summary report.

The summary will be displayed at the end of .eval().

pub fn build<M>( self, model: M, ) -> Evaluator<EvaluatorComponentTypesMarker<B, M, AsyncProcessorEvaluation<FullEventProcessorEvaluation<TO>>, TI, TO>>

where TI: Send + 'static, M: Module<B> + TestStep<TI, TO> + Display + 'static,

Builds the evaluator.