Activation

burn::nn::activation

Enum Activation

#[non_exhaustive]
pub enum Activation<B>where
    B: Backend,{
Show 18 variants
    Gelu(Gelu),
    PRelu(PRelu<B>),
    Relu(Relu),
    LeakyRelu(LeakyRelu),
    SwiGlu(SwiGlu<B>),
    Selu(Selu),
    Sigmoid(Sigmoid),
    Tanh(Tanh),
    HardSigmoid(HardSigmoid),
    HardSwish(HardSwish),
    Softplus(Softplus),
    Softsign(Softsign),
    Elu(Elu),
    Celu(Celu),
    ThresholdedRelu(ThresholdedRelu),
    HardShrink(HardShrink),
    SoftShrink(SoftShrink),
    Shrink(Shrink),
}

Expand description

Activation Layer Wrapper.

Provides support for many in-built burn::nn activations.

Variants (Non-exhaustive)§

This enum is marked as non-exhaustive

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Gelu(Gelu)

Gelu activation layer.

PRelu(PRelu<B>)

PRelu activation layer.

Relu(Relu)

Relu activation layer.

LeakyRelu(LeakyRelu)

LeakyRelu activation layer.

SwiGlu(SwiGlu<B>)

SwiGlu activation layer.

Selu(Selu)

Selu activation layer.

Sigmoid(Sigmoid)

Sigmoid activation layer.

Tanh(Tanh)

Tanh activation layer.

HardSigmoid(HardSigmoid)

HardSigmoid activation layer.

HardSwish(HardSwish)

HardSwish activation layer.

Softplus(Softplus)

Softplus activation layer.

Softsign(Softsign)

Softsign activation layer.

Elu(Elu)

Elu activation layer.

Celu(Celu)

Celu activation layer.

ThresholdedRelu(ThresholdedRelu)

ThresholdedRelu activation layer.

HardShrink(HardShrink)

HardShrink activation layer.

SoftShrink(SoftShrink)

SoftShrink activation layer.

Shrink(Shrink)

Shrink activation layer.

Implementations§

impl<B> Activation<B>
where B: Backend,

pub fn forward<const D: usize>(&self, input: Tensor<B, D>) -> Tensor<B, D>

Forward pass.

Trait Implementations§

impl<B> AutodiffModule<B> for Activation<B>
where B: Backend + AutodiffBackend, <B as AutodiffBackend>::InnerBackend: Backend,

type InnerModule = Activation<<B as AutodiffBackend>::InnerBackend>

Inner module without auto-differentiation.

fn valid(&self) -> <Activation<B> as AutodiffModule<B>>::InnerModule

Returns the same module, but on the inner backend without auto-differentiation.

fn from_inner( module: <Activation<B> as AutodiffModule<B>>::InnerModule, ) -> Activation<B>

Wraps an inner module back into an auto-diff module.

impl<B> Clone for Activation<B>
where B: Backend,

fn clone(&self) -> Activation<B>

Returns a duplicate of the value. Read more

1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more

impl<B> Debug for Activation<B>
where B: Debug + Backend,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

impl<B> Display for Activation<B>
where B: Backend,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

impl<B> From<Celu> for Activation<B>
where B: Backend,

fn from(layer: Celu) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Elu> for Activation<B>
where B: Backend,

fn from(layer: Elu) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Gelu> for Activation<B>
where B: Backend,

fn from(layer: Gelu) -> Activation<B>

Converts to this type from the input type.

impl<B> From<HardShrink> for Activation<B>
where B: Backend,

fn from(layer: HardShrink) -> Activation<B>

Converts to this type from the input type.

impl<B> From<HardSigmoid> for Activation<B>
where B: Backend,

fn from(layer: HardSigmoid) -> Activation<B>

Converts to this type from the input type.

impl<B> From<HardSwish> for Activation<B>
where B: Backend,

fn from(layer: HardSwish) -> Activation<B>

Converts to this type from the input type.

impl<B> From<LeakyRelu> for Activation<B>
where B: Backend,

fn from(layer: LeakyRelu) -> Activation<B>

Converts to this type from the input type.

impl<B> From<PRelu<B>> for Activation<B>
where B: Backend,

fn from(layer: PRelu<B>) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Relu> for Activation<B>
where B: Backend,

fn from(layer: Relu) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Selu> for Activation<B>
where B: Backend,

fn from(layer: Selu) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Shrink> for Activation<B>
where B: Backend,

fn from(layer: Shrink) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Sigmoid> for Activation<B>
where B: Backend,

fn from(layer: Sigmoid) -> Activation<B>

Converts to this type from the input type.

impl<B> From<SoftShrink> for Activation<B>
where B: Backend,

fn from(layer: SoftShrink) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Softplus> for Activation<B>
where B: Backend,

fn from(layer: Softplus) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Softsign> for Activation<B>
where B: Backend,

fn from(layer: Softsign) -> Activation<B>

Converts to this type from the input type.

impl<B> From<SwiGlu<B>> for Activation<B>
where B: Backend,

fn from(layer: SwiGlu<B>) -> Activation<B>

Converts to this type from the input type.

impl<B> From<Tanh> for Activation<B>
where B: Backend,

fn from(layer: Tanh) -> Activation<B>

Converts to this type from the input type.

impl<B> From<ThresholdedRelu> for Activation<B>
where B: Backend,

fn from(layer: ThresholdedRelu) -> Activation<B>

Converts to this type from the input type.

impl<B> HasAutodiffModule<B> for Activation<<B as AutodiffBackend>::InnerBackend>
where B: Backend + AutodiffBackend, <B as AutodiffBackend>::InnerBackend: Backend,

type TrainModule = Activation<B>

The module with auto-differentiation.

impl<B> Module<B> for Activation<B>
where B: Backend,

type Record = ActivationRecord<B>

Type to save and load the module.

fn load_record( self, record: <Activation<B> as Module<B>>::Record, ) -> Activation<B>

Load the module state from a record.

fn into_record(self) -> <Activation<B> as Module<B>>::Record

Convert the module into a record containing the state.

fn num_params(&self) -> usize

Get the number of parameters the module has, including all of its sub-modules.

fn visit<Visitor>(&self, visitor: &mut Visitor)
where Visitor: ModuleVisitor<B>,

Visit each tensor parameter in the module with a visitor.

fn map<Mapper>(self, mapper: &mut Mapper) -> Activation<B>
where Mapper: ModuleMapper<B>,

Map each tensor parameter in the module with a mapper.

fn collect_devices( &self, devices: Vec<<B as BackendTypes>::Device>, ) -> Vec<<B as BackendTypes>::Device>

Return all the devices found in the underneath module tree added to the given vector without duplicates.

fn to_device(self, device: &<B as BackendTypes>::Device) -> Activation<B>

Move the module and all of its sub-modules to the given device. Read more

fn fork(self, device: &<B as BackendTypes>::Device) -> Activation<B>

Fork the module and all of its sub-modules to the given device. Read more

fn devices(&self) -> Vec<<B as BackendTypes>::Device>

Return all the devices found in the underneath module tree without duplicates.

fn no_grad(self) -> Self

Each tensor in the module tree will not require grad. Read more

fn train<AB>(self) -> Self::TrainModule
where AB: AutodiffBackend<InnerBackend = B>, Self: HasAutodiffModule<AB>,

Move the module and all of its sub-modules to the autodiff backend. Read more

fn save_file<FR, PB>( self, file_path: PB, recorder: &FR, ) -> Result<(), RecorderError>
where FR: FileRecorder<B>, PB: Into<PathBuf>,

Save the module to a file using the provided file recorder. Read more

fn load_file<FR, PB>( self, file_path: PB, recorder: &FR, device: &<B as BackendTypes>::Device, ) -> Result<Self, RecorderError>
where FR: FileRecorder<B>, PB: Into<PathBuf>,

Load the module from a file using the provided file recorder. Read more

fn quantize_weights(self, quantizer: &mut Quantizer) -> Self

Quantize the weights of the module.

impl<B> ModuleDisplay for Activation<B>
where B: Backend,

fn format(&self, passed_settings: DisplaySettings) -> String

Formats the module with provided display settings. Read more

fn custom_settings(&self) -> Option<DisplaySettings>

Custom display settings for the module. Read more

fn custom_content(&self, _content: Content) -> Option<Content>

Custom attributes for the module. Read more

impl<B> ModuleDisplayDefault for Activation<B>
where B: Backend,

fn content(&self, content: Content) -> Option<Content>

Attributes of the module used for display purposes. Read more

fn num_params(&self) -> usize

Gets the number of the parameters of the module.

Auto Trait Implementations§

impl<B> !Freeze for Activation<B>

impl<B> !RefUnwindSafe for Activation<B>

impl<B> Send for Activation<B>

impl<B> Sync for Activation<B>

impl<B> Unpin for Activation<B>
where <B as BackendTypes>::FloatTensorPrimitive: Unpin, <B as BackendTypes>::QuantizedTensorPrimitive: Unpin, <B as BackendTypes>::Device: Unpin,

impl<B> !UnwindSafe for Activation<B>

Blanket Implementations§

impl<T> Adaptor<()> for T

fn adapt(&self)

Adapt the type to be passed to a metric.

impl<T> Any for T
where T: 'static + ?Sized,

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more

impl<T> Borrow<T> for T
where T: ?Sized,

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more

impl<T> BorrowMut<T> for T
where T: ?Sized,

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more

impl<C> CloneExpand for C
where C: Clone,

fn __expand_clone_method(&self, _scope: &mut Scope) -> C

impl<T> CloneToUninit for T
where T: Clone,

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)

Performs copy-assignment from self to dest. Read more

impl<T> Downcast<T> for T

fn downcast(&self) -> &T

impl<T> From<T> for T

fn from(t: T) -> T

Returns the argument unchanged.

impl<T> Instrument for T

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more

impl<T, U> Into<U> for T
where U: From<T>,

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

impl<T> IntoComptime for T

fn comptime(self) -> Self

impl<T> IntoEither for T

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more

impl<T> Pointable for T

const ALIGN: usize

The alignment of pointer.

type Init = T

The type for initializers.

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more

impl<T> ToCompactString for T
where T: Display,

fn try_to_compact_string(&self) -> Result<CompactString, ToCompactStringError>

Fallible version of [ToCompactString::to_compact_string()] Read more

fn to_compact_string(&self) -> CompactString

Converts the given value to a [CompactString]. Read more

impl<T> ToLine for T
where T: Display,

fn to_line(&self) -> Line<'_>

Converts the value to a [Line].

impl<T> ToOwned for T
where T: Clone,

type Owned = T

The resulting type after obtaining ownership.

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more

impl<T> ToSpan for T
where T: Display,

fn to_span(&self) -> Span<'_>

Converts the value to a [Span].

impl<T> ToString for T
where T: Display + ?Sized,

fn to_string(&self) -> String

Converts the given value to a String. Read more

impl<T> ToText for T
where T: Display,

fn to_text(&self) -> Text<'_>

Converts the value to a [Text].

impl<T, U> TryFrom<U> for T
where U: Into<T>,

type Error = Infallible

The type returned in the event of a conversion error.

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.

impl<T> TuneInputs for T
where T: Clone + Send + Sync + 'static,

type At<'a> = T

The concrete input type at lifetime 'a.

impl<T> Upcast<T> for T

fn upcast(&self) -> Option<&T>

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

fn vzip(self) -> V

impl<T> WithSubscriber for T

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more

impl<T> WasmNotSend for T
where T: Send,

impl<T> WasmNotSendSync for T
where T: WasmNotSend + WasmNotSync,

impl<T> WasmNotSync for T
where T: Sync,