Skip to main content

gelu_approximate

Function gelu_approximate 

pub fn gelu_approximate<const D: usize>(tensor: Tensor<D>) -> Tensor<D>
Expand description

Applies the tanh-based approximate GELU function element-wise.

GELU_approx(x) = 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))