Function generate_sinusoids 

pub fn generate_sinusoids(
    length: usize,
    d_model: usize,
    max_timescale: usize,
    device: &Device,
) -> Tensor<2>

Returns sinusoids for positional embedding introduced in Attention is all you need.

The reference implementation can be found here: LANGUAGE MODELING WITH NN.TRANSFORMER AND TORCHTEXT

Arguments

• length - The length of the sequence.
• d_model - The size of each vector.
• max_timescale - The maximum time scale to use.

Returns

A tensor of shape [length, d_model] containing the sinusoids.