Struct AttentionModuleOptions 

pub struct AttentionModuleOptions {
    pub scale: Option<f64>,
    pub softcap: Option<f64>,
    pub is_causal: bool,
}

Options for attention.

Fields

scale: Option<f64>

Custom scale factor applied to QK^T. When None, defaults to 1/sqrt(head_dim).

softcap: Option<f64>

Soft capping applied before softmax: softcap * tanh(scores / softcap). Used by Gemma-2 and similar models. Must be positive when set.

is_causal: bool

When true, applies causal (autoregressive) masking so that each query position can only attend to key positions at or before it. This is more efficient than passing an explicit lower-triangular bool mask because backends can use optimized kernel paths (e.g. flash attention with causal mode).

Trait Implementations

impl Clone for AttentionModuleOptions

fn clone(&self) -> AttentionModuleOptions

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttentionModuleOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for AttentionModuleOptions

fn default() -> AttentionModuleOptions

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for AttentionModuleOptions

fn deserialize<__D>( __deserializer: __D, ) -> Result<AttentionModuleOptions, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<AttentionModuleOptions> for AttentionOptionsIr

fn from(ir: AttentionModuleOptions) -> AttentionOptionsIr

Converts to this type from the input type.

impl From<AttentionOptionsIr> for AttentionModuleOptions

fn from(ir: AttentionOptionsIr) -> AttentionModuleOptions

Converts to this type from the input type.

impl PartialEq for AttentionModuleOptions

fn eq(&self, other: &AttentionModuleOptions) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for AttentionModuleOptions

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for AttentionModuleOptions

impl StructuralPartialEq for AttentionModuleOptions