Struct HandleContainer 

pub struct HandleContainer<H> { /* private fields */ }

Keep all tensor handles in one place and ensure that all resources are used optimally.

Implementations

impl<H> HandleContainer<H>

where H: Clone,

pub fn fork(&self) -> HandleContainer<H>

Fork the container, useful for autotune.

impl<H> HandleContainer<H>

where H: Clone,

pub fn new() -> HandleContainer<H>

Create a new HandleContainer

pub fn has_errors(&self) -> bool

Whether any tensor is errored by a failure.

A branch that lets the error checks cost nothing while nothing has failed. False means no error lookup can find anything, so the caller can skip asking — worth a branch where asking costs a boxed iterator over an operation’s inputs.

pub fn register_handle(&mut self, id: TensorId, handle: H)

Register a handle for the given tensor id.

Writing recovers a tensor a failure claimed: the bytes are there now, so the entry stops being Handle::Errored and reads of it succeed again. Every register_* path goes through the same insert, so this holds however the write arrives.

This is what lets work retry. An autotune candidate that fails claims the output it did not write; the next candidate writes it and clears the claim, and execution carries on with nothing downstream skipped.

pub fn has_handle(&self, id: &TensorId) -> bool

Whether a usable handle exists.

False for a tensor claimed by a failure (Handle::Errored): the entry is there, but there is no data behind it.

pub fn get_handle_ref(&self, id: &TensorId) -> Option<&H>

Get the reference to a handle.

None for a tensor claimed by a failure, the same as one that was never produced — the error is delivered by get_handle, at the read, and asked for ahead of time with error.

pub fn set_error(&mut self, id: TensorId, error: TensorError)

Record that error is why id holds no data: the work that was going to write it did not run, so a read of it must fail rather than hand back whatever the tensor id resolves to.

It always displaces whatever is registered. A claim covers exactly the tensors one unit of work was responsible for writing, and a handle registered before that work — an in-place output, aliased to its input while the launch is still being planned — proves nothing about whether the kernel that fills it ever ran.

pub fn claim_unwritten(&mut self, id: TensorId, error: TensorError)

Claim id, but only if nothing wrote it and no failure already claims it.

For the one caller that cannot name its write set: a panic that escaped the strategy walk itself, raised outside every scope, where all that is known is that something in the consumed segment did not finish. Claiming broadly from there would error outputs that other operations wrote, and would overwrite a precise claim with a vaguer one — so it claims only what is neither, which is exactly the set nothing else can account for.

pub fn error(&self, id: &TensorId) -> Option<&TensorError>

The failure that errored id, if one did.

pub fn take_error(&mut self, tensor: &TensorIr) -> Option<TensorError>

The failure tensor carries, released when this read consumes the tensor. The fallible half of get_handle: a read asks for this first and reports it, rather than tripping over the handle that is not there.

The status rule is the one get_handle already applies to a handle. A ReadWrite read is the tensor’s last use, so the failure is released with it — the refcount that clears failures is the tensor’s own, and this read is the last chance to honour it: the reader took ownership of the id, so no later Drop will. A ReadOnly read leaves it, because the tensor is still alive and every later read of it has to report the same cause.

pub fn get_handle(&mut self, id: &TensorId, status: &TensorStatus) -> H

Get the handle for the given tensor id. The status is used to determine if the tensor should be popped out of the current tensor map, necessary for inplace operations.

Warnings

Make sure the status corresponds to the operation you want to execute the handle on, otherwise you might remove a tensor handle that will be required in the future.

pub fn get_tensor_handle(&mut self, tensor: &TensorIr) -> TensorHandle<H>

Get the tensor handle for the given tensor intermediate representation.

pub fn get_float_tensor<B>( &mut self, tensor: &TensorIr, ) -> <B as BackendTypes>::FloatTensorPrimitive

where B: BackendIr<Handle = H>,

Get the float tensor corresponding to the given tensor intermediate representation.

pub fn get_int_tensor<B>( &mut self, tensor: &TensorIr, ) -> <B as BackendTypes>::IntTensorPrimitive

where B: BackendIr<Handle = H>,

Get the int tensor corresponding to the given tensor intermediate representation.

pub fn get_bool_tensor<B>( &mut self, tensor: &TensorIr, ) -> <B as BackendTypes>::BoolTensorPrimitive

where B: BackendIr<Handle = H>,

Get the bool tensor corresponding to the given tensor intermediate representation.

pub fn get_quantized_tensor<B>( &mut self, tensor: &TensorIr, ) -> <B as BackendTypes>::QuantizedTensorPrimitive

where B: BackendIr<Handle = H>,

Get the quantized tensor corresponding to the given tensor intermediate representation.

pub fn register_float_tensor<B>( &mut self, id: &TensorId, tensor: <B as BackendTypes>::FloatTensorPrimitive, )

where B: BackendIr<Handle = H>,

Register a new float tensor with the corresponding tensor id.

pub fn register_quantized_tensor<B>( &mut self, id: &TensorId, tensor: <B as BackendTypes>::QuantizedTensorPrimitive, )

where B: BackendIr<Handle = H>,

Register a new quantized tensor with the corresponding tensor ids.

pub fn register_int_tensor<B>( &mut self, id: &TensorId, tensor: <B as BackendTypes>::IntTensorPrimitive, )

where B: BackendIr<Handle = H>,

Register a new int tensor with the corresponding tensor id.

pub fn register_bool_tensor<B>( &mut self, id: &TensorId, tensor: <B as BackendTypes>::BoolTensorPrimitive, )

where B: BackendIr<Handle = H>,

Register a new bool tensor with the corresponding tensor id.

pub fn remove_handle(&mut self, id: TensorId) -> Option<Handle<H>>

Remove tensor handle from container.

pub fn free(&mut self, tensor: &TensorIr)

Remove tensor handle from container if writable

pub fn num_handles(&self) -> usize

Returns the number of handles.

pub fn handle_ids(&self) -> impl Iterator<Item = &TensorId>

Returns the IDs of all currently registered handles.

Useful for snapshotting which handles exist at a point in time (e.g., before executing on a forked context) so that newly registered output handles can be detected afterwards.

Trait Implementations

impl<H> Debug for HandleContainer<H>

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<H> Default for HandleContainer<H>

fn default() -> HandleContainer<H>

Returns the “default value” for a type.