Struct LambConfig 

pub struct LambConfig { /* private fields */ }

Lamb configuration.

Implementations

impl LambConfig

pub fn new() -> LambConfig

Create a new instance of the config.

Arguments

Optional Arguments

grad_clipping

Optional gradient clipping configuration.

• Defaults to None

Default Arguments

beta_1

Exponential decay rate for the first moment estimates.

• Defaults to 0.9

beta_2

Exponential decay rate for the second moment estimates.

• Defaults to 0.999

epsilon

A value added to the denominator for numerical stability.

• Defaults to 1e-6

weight_decay

Weight decay applied to the Adam update before layer-wise adaptation.

• Defaults to 0.0

use_trust_ratio

Whether to scale each parameter update by its layer-wise trust ratio.

• Defaults to true

impl LambConfig

pub fn with_beta_1(self, beta_1: f32) -> LambConfig

Sets the value for the field beta_1.

Exponential decay rate for the first moment estimates.

• Defaults to 0.9

pub fn with_beta_2(self, beta_2: f32) -> LambConfig

Sets the value for the field beta_2.

Exponential decay rate for the second moment estimates.

• Defaults to 0.999

pub fn with_epsilon(self, epsilon: f32) -> LambConfig

Sets the value for the field epsilon.

A value added to the denominator for numerical stability.

• Defaults to 1e-6

pub fn with_weight_decay(self, weight_decay: f32) -> LambConfig

Sets the value for the field weight_decay.

Weight decay applied to the Adam update before layer-wise adaptation.

• Defaults to 0.0

pub fn with_use_trust_ratio(self, use_trust_ratio: bool) -> LambConfig

Sets the value for the field use_trust_ratio.

Whether to scale each parameter update by its layer-wise trust ratio.

• Defaults to true

pub fn with_grad_clipping( self, grad_clipping: Option<GradientClippingConfig>, ) -> LambConfig

Sets the value for the field grad_clipping.

Optional gradient clipping configuration.

• Defaults to None

impl LambConfig

pub fn build(&self) -> Lamb

Build the per-parameter LAMB optimizer.

Use Self::init to construct a whole-module optimizer with the configured gradient clipping behavior.

pub fn init(&self) -> ModuleOptimizer

Initialize a whole-module LAMB optimizer.

Trait Implementations

impl Clone for LambConfig

fn clone(&self) -> LambConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for LambConfig

fn save<P>(&self, file: P) -> Result<(), Error>

where P: AsRef<Path>,

Saves the configuration to a file.

fn load<P>(file: P) -> Result<Self, ConfigError>

where P: AsRef<Path>,

Loads the configuration from a file.

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for LambConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for LambConfig

fn deserialize<D>( deserializer: D, ) -> Result<LambConfig, <D as Deserializer<'de>>::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for LambConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Serialize for LambConfig

fn serialize<S>( &self, serializer: S, ) -> Result<<S as Serializer>::Ok, <S as Serializer>::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.