Struct SwiGluConfig 

pub struct SwiGluConfig {
    pub d_input: usize,
    pub d_output: usize,
    pub bias: bool,
    pub initializer: Initializer,
    pub layout: LinearLayout,
}

Configuration to create a SwiGlu activation layer using the init function.

Fields

d_input: usize

The size of the input features.

d_output: usize

The size of the output features.

bias: bool

If a bias should be applied during the linear transformation. Default behaviour is False for SwiGLU activation implementations.

initializer: Initializer

The type of function used to initialize the linear layer parameters

layout: LinearLayout

The layout in which the linear parameters are stored.

Implementations

impl SwiGluConfig

pub fn new(d_input: usize, d_output: usize) -> SwiGluConfig

Create a new instance of the config.

Arguments

Required Arguments

d_input

The size of the input features.

d_output

The size of the output features.

Default Arguments

bias

If a bias should be applied during the linear transformation. Default behaviour is False for SwiGLU activation implementations.

• Defaults to false

initializer

The type of function used to initialize the linear layer parameters

• Defaults to "Initializer::KaimingUniform{gain:1.0/num_traits::Float::sqrt(3.0), fan_out_only:false}"

layout

The layout in which the linear parameters are stored.

• Defaults to "LinearLayout::Row"

impl SwiGluConfig

pub fn with_bias(self, bias: bool) -> SwiGluConfig

Sets the value for the field bias.

If a bias should be applied during the linear transformation. Default behaviour is False for SwiGLU activation implementations.

• Defaults to false

pub fn with_initializer(self, initializer: Initializer) -> SwiGluConfig

Sets the value for the field initializer.

The type of function used to initialize the linear layer parameters

• Defaults to "Initializer::KaimingUniform{gain:1.0/num_traits::Float::sqrt(3.0), fan_out_only:false}"

pub fn with_layout(self, layout: LinearLayout) -> SwiGluConfig

Sets the value for the field layout.

The layout in which the linear parameters are stored.

• Defaults to "LinearLayout::Row"

impl SwiGluConfig

pub fn init(&self, device: &Device) -> SwiGlu

Initialize a new SwiGLU activation layer.

Trait Implementations

impl Clone for SwiGluConfig

fn clone(&self) -> SwiGluConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for SwiGluConfig

fn save<P>(&self, file: P) -> Result<(), Error>

where P: AsRef<Path>,

Saves the configuration to a file.

fn load<P>(file: P) -> Result<Self, ConfigError>

where P: AsRef<Path>,

Loads the configuration from a file.

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for SwiGluConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for SwiGluConfig

fn deserialize<D>( deserializer: D, ) -> Result<SwiGluConfig, <D as Deserializer<'de>>::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for SwiGluConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl From<SwiGluConfig> for ActivationConfig

fn from(config: SwiGluConfig) -> ActivationConfig

Converts to this type from the input type.

impl Serialize for SwiGluConfig

fn serialize<S>( &self, serializer: S, ) -> Result<<S as Serializer>::Ok, <S as Serializer>::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.