Function gelu

pub fn gelu<const D: usize, B>(tensor: Tensor<B, D>) -> Tensor<B, D>
where B: Backend,
Expand description

Applies the Gaussian Error Linear Units function as described in the paper Gaussian Error Linear Units (GELUs).

GELU(x) = x * Φ(x) = x * 1/2 * (1 + erf(x / sqrt(2)))

where Φ(x) is the cumulative distribution function for the Gaussian distribution.