Struct PositionWiseFeedForwardConfig 

pub struct PositionWiseFeedForwardConfig {
    pub d_model: usize,
    pub d_ff: usize,
    pub dropout: f64,
    pub initializer: Initializer,
    pub activation: ActivationConfig,
}

Configuration to create a position-wise feed-forward layer using the init function.

Fields

d_model: usize

The size of the input and output features.

d_ff: usize

The size of the hidden inner features.

dropout: f64

The dropout rate. Default: 0.1

initializer: Initializer

The type of function used to initialize neural network parameters

activation: ActivationConfig

The activation function used between the two linear layers. Default: Gelu

Implementations

impl PositionWiseFeedForwardConfig

pub fn new(d_model: usize, d_ff: usize) -> PositionWiseFeedForwardConfig

Create a new instance of the config.

Arguments

Required Arguments

d_model

The size of the input and output features.

d_ff

The size of the hidden inner features.

Default Arguments

dropout

The dropout rate. Default: 0.1

• Defaults to 0.1

initializer

The type of function used to initialize neural network parameters

• Defaults to "Initializer::KaimingUniform{gain:1.0/num_traits::Float::sqrt(3.0), fan_out_only:false}"

activation

The activation function used between the two linear layers. Default: Gelu

• Defaults to "ActivationConfig::Gelu"

impl PositionWiseFeedForwardConfig

pub fn with_dropout(self, dropout: f64) -> PositionWiseFeedForwardConfig

Sets the value for the field dropout.

The dropout rate. Default: 0.1

• Defaults to 0.1

pub fn with_initializer( self, initializer: Initializer, ) -> PositionWiseFeedForwardConfig

Sets the value for the field initializer.

The type of function used to initialize neural network parameters

• Defaults to "Initializer::KaimingUniform{gain:1.0/num_traits::Float::sqrt(3.0), fan_out_only:false}"

pub fn with_activation( self, activation: ActivationConfig, ) -> PositionWiseFeedForwardConfig

Sets the value for the field activation.

The activation function used between the two linear layers. Default: Gelu

• Defaults to "ActivationConfig::Gelu"

impl PositionWiseFeedForwardConfig

pub fn init<B>( &self, device: &<B as BackendTypes>::Device, ) -> PositionWiseFeedForward<B>

where B: Backend,

Initialize a new position-wise feed-forward module.

Trait Implementations

impl Clone for PositionWiseFeedForwardConfig

fn clone(&self) -> PositionWiseFeedForwardConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for PositionWiseFeedForwardConfig

fn save<P>(&self, file: P) -> Result<(), Error>

where P: AsRef<Path>,

Saves the configuration to a file.

fn load<P>(file: P) -> Result<Self, ConfigError>

where P: AsRef<Path>,

Loads the configuration from a file.

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for PositionWiseFeedForwardConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for PositionWiseFeedForwardConfig

fn deserialize<D>( deserializer: D, ) -> Result<PositionWiseFeedForwardConfig, <D as Deserializer<'de>>::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for PositionWiseFeedForwardConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Serialize for PositionWiseFeedForwardConfig

fn serialize<S>( &self, serializer: S, ) -> Result<<S as Serializer>::Ok, <S as Serializer>::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.