Struct CrossAttentionCache
pub struct CrossAttentionCache<B>where
B: Backend,{
pub k: TensorCache<B, 4>,
pub v: TensorCache<B, 4>,
}Expand description
Cache for the Cross Attention layer.
To be used during inference when context is constant.
Fields§
§k: TensorCache<B, 4>Cached key tensor.
v: TensorCache<B, 4>Cached value tensor.
Implementations§
§impl<B> CrossAttentionCache<B>where
B: Backend,
impl<B> CrossAttentionCache<B>where
B: Backend,
pub fn new() -> CrossAttentionCache<B>
pub fn new() -> CrossAttentionCache<B>
Create a new empty cache.
Trait Implementations§
§impl<B> Default for CrossAttentionCache<B>where
B: Backend,
impl<B> Default for CrossAttentionCache<B>where
B: Backend,
§fn default() -> CrossAttentionCache<B>
fn default() -> CrossAttentionCache<B>
Returns the “default value” for a type. Read more
Auto Trait Implementations§
impl<B> Freeze for CrossAttentionCache<B>where
<B as Backend>::FloatTensorPrimitive: Freeze,
<B as Backend>::QuantizedTensorPrimitive: Freeze,
impl<B> RefUnwindSafe for CrossAttentionCache<B>where
<B as Backend>::FloatTensorPrimitive: RefUnwindSafe,
<B as Backend>::QuantizedTensorPrimitive: RefUnwindSafe,
impl<B> Send for CrossAttentionCache<B>
impl<B> Sync for CrossAttentionCache<B>
impl<B> Unpin for CrossAttentionCache<B>
impl<B> UnwindSafe for CrossAttentionCache<B>where
<B as Backend>::FloatTensorPrimitive: UnwindSafe,
<B as Backend>::QuantizedTensorPrimitive: UnwindSafe,
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more