Struct AdamW
pub struct AdamW { /* private fields */ }Expand description
AdamW optimizer.
See:
- Decoupled Weight Decay Regularization, Loshchilov and Hutter, 2019.
- Cautious Weight Decay, 2025
- On the Convergence of Adam and Beyond
Configured by AdamWConfig.
Trait Implementations§
§impl<B> SimpleOptimizer<B> for AdamWwhere
B: Backend,
impl<B> SimpleOptimizer<B> for AdamWwhere
B: Backend,
§fn step<const D: usize>(
&self,
lr: f64,
tensor: Tensor<B, D>,
grad: Tensor<B, D>,
state: Option<<AdamW as SimpleOptimizer<B>>::State<D>>,
) -> (Tensor<B, D>, Option<<AdamW as SimpleOptimizer<B>>::State<D>>)
fn step<const D: usize>( &self, lr: f64, tensor: Tensor<B, D>, grad: Tensor<B, D>, state: Option<<AdamW as SimpleOptimizer<B>>::State<D>>, ) -> (Tensor<B, D>, Option<<AdamW as SimpleOptimizer<B>>::State<D>>)
A single optimization step for any tensor that represents the parameters of a model.
§type State<const D: usize> = AdamWState<B, D>
type State<const D: usize> = AdamWState<B, D>
The state of the optimizer. It also implements record, so that it can be saved.
§fn to_device<const D: usize>(
state: <AdamW as SimpleOptimizer<B>>::State<D>,
device: &<B as BackendTypes>::Device,
) -> <AdamW as SimpleOptimizer<B>>::State<D>
fn to_device<const D: usize>( state: <AdamW as SimpleOptimizer<B>>::State<D>, device: &<B as BackendTypes>::Device, ) -> <AdamW as SimpleOptimizer<B>>::State<D>
Change the device of the state. Read more
Auto Trait Implementations§
impl Freeze for AdamW
impl RefUnwindSafe for AdamW
impl Send for AdamW
impl Sync for AdamW
impl Unpin for AdamW
impl UnwindSafe for AdamW
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
§impl<C> CloneExpand for Cwhere
C: Clone,
impl<C> CloneExpand for Cwhere
C: Clone,
fn __expand_clone_method(&self, _scope: &mut Scope) -> C
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more