Module activation 

The activation module.

Functions

celu

Applies the Continuously Differentiable Exponential Linear Unit function element-wise.

elu

Applies the Exponential Linear Unit function element-wise.

gelu

Applies the Gaussian Error Linear Units function as described in the paper Gaussian Error Linear Units (GELUs).

gelu_approximate

Applies the tanh-based approximate GELU function element-wise.

glu

Applies the gated linear unit function.

hard_shrink

Applies the HardShrink function element-wise.

hard_sigmoid

Applies the hard sigmoid function element-wise.

hard_swish

Applies the hard swish function element-wise.

leaky_relu

Applies the leaky rectified linear unit function element-wise.

log_sigmoid

Applies the log sigmoid function element-wise.

log_softmax

Applies the log softmax function on the input tensor along the given dimension.

mish

Applies the Mish function as described in the paper in Mish: A Self Regularized Non-Monotonic Neural Activation Function.

prelu

Applies Parametric ReLu activation function as described in the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.

quiet_softmax

Applies the “quiet softmax” function on the input tensor along the given dimension.

relu

Applies the rectified linear unit function element-wise as described in the paper Deep Learning using Rectified Linear Units (ReLU).

selu

Applies the Scaled Exponential Linear Unit function element-wise as described in the paper Self-Normalizing Neural Networks.

shrink

Applies the Shrink function element-wise.

sigmoid

Applies the sigmoid function element-wise.

silu

Applies the SiLU function (also known as the swish function) element-wise.

soft_shrink

Applies the SoftShrink function element-wise.

softmax

Applies the softmax function on the input tensor along the given dimension.

softmin

Applies the softmin function on the input tensor along the given dimension.

softplus

Applies the SoftPlus function element-wise.

softsign

Applies the Softsign function element-wise.

tanh

Applies the tanh function element-wise.

thresholded_relu

Applies the thresholded rectified linear unit function element-wise.