Skip to main content

glu

Function glu 

pub fn glu<const D: usize>(tensor: Tensor<D>, dim: impl AsIndex) -> Tensor<D>
Expand description

Applies the gated linear unit function.

GLU(a,b)=a⊗σ(b) where a is the first half of the input matrices and b is the second half.

Note:

  • The size of the input tensor along dim must be divisible by 2.
  • Negative dimensions are supported and count from the end.

§Arguments

  • tensor - The input tensor.
  • dim - The dimension on which to split the input.

§Returns

  • A tensor with the same shape as the input, except the size along dim is halved.