Struct GlobalFusionClient 

pub struct GlobalFusionClient<R>
where
    R: FusionRuntime,
{ /* private fields */ }

Use a mutex to communicate with the fusion server.

Implementations

impl<R> GlobalFusionClient<R>

where R: FusionRuntime + 'static,

pub fn load( device: &<R as FusionRuntime>::FusionDevice, ) -> GlobalFusionClient<R>

Loads the client from the given device.

impl<R> GlobalFusionClient<R>

where R: FusionRuntime + 'static,

pub fn new(device: <R as FusionRuntime>::FusionDevice) -> GlobalFusionClient<R>

Create a new client for the given device.

pub fn register<O>( &self, stream: StreamId, repr: OperationIr, operation: O, ) -> Vec<FusionTensor<R>>

where O: Operation<R> + 'static,

Register a new tensor operation intermediate representation on stream.

Returns the new (uninitialized) output tensor(s) generated by the registered operation.

Cross-stream tensor sharing is resolved out-of-band by tag_shared_view before this call (see MultiStream docs for the full strategy), so every input tensor in repr is assumed to live on stream. Pass StreamId::current() for normal ops and the tensor’s own stream for Drop.

pub fn run<Re>(&self, func: impl FnOnce() -> Re + Send) -> Result<Re, CallError>

where Re: Send,

Run func on the server, in order with the operations registered so far but without executing the ones still queued — sync is the one that drains them first.

The server runs a task where the calling stream stands, so this is how a caller marks a point in the stream’s execution without cutting the queue at it.

Errors

The server could not run the task: it panicked, or is gone.

pub fn sync<Re>(&self, sync_fn: impl FnOnce() -> Re + Send + 'static) -> Re

where Re: Send + 'static,

Register all lazy computation.

Panics

When the server could not run the task, re-raising its own panic so the original cause is what the caller sees. try_sync returns it instead.

pub fn try_sync<Re>( &self, sync_fn: impl FnOnce() -> Re + Send + 'static, ) -> Result<Re, CallError>

where Re: Send + 'static,

Register all lazy computation.

Errors

The server could not run the task: it panicked, or is gone.

pub fn flush_queue(&self)

Flush the operations queue.

pub fn create_empty_handle(&self) -> TensorId

Create a new (uninitialized) empty tensor handle and returns its corresponding tensor id.

pub fn device(&self) -> &<R as FusionRuntime>::FusionDevice

Get the current device used by all operations handled by this client.

pub fn register_tensor_handle( &self, handle: <R as FusionRuntime>::FusionHandle, ) -> TensorId

Create a tensor with the given handle and returns its corresponding tensor id.

pub fn read_tensor_float<B>( self, tensor: TensorIr, stream: StreamId, ) -> impl Future<Output = Result<TensorData, ExecutionError>> + Send

where B: FusionBackend<FusionRuntime = R>,

Read the values contained by a float tensor.

pub fn read_tensor_int<B>( self, tensor: TensorIr, stream: StreamId, ) -> impl Future<Output = Result<TensorData, ExecutionError>> + Send

where B: FusionBackend<FusionRuntime = R>,

Read the values contained by an int tensor.

pub fn read_tensor_bool<B>( self, tensor: TensorIr, stream: StreamId, ) -> impl Future<Output = Result<TensorData, ExecutionError>> + Send

where B: FusionBackend<FusionRuntime = R>,

Read the values contained by a bool tensor.

pub fn read_tensor_quantized<B>( self, tensor: TensorIr, stream: StreamId, ) -> impl Future<Output = Result<TensorData, ExecutionError>> + Send

where B: FusionBackend<FusionRuntime = R>,

Read the values contained by a quantized tensor.

pub fn change_client_float<B>( tensor: TensorIr, client_src: GlobalFusionClient<R>, client_dst: GlobalFusionClient<R>, stream: StreamId, ) -> FusionTensor<R>

where B: FusionBackend<FusionRuntime = R>,

Change the client of the given float tensor.

pub fn change_client_int<B>( tensor: TensorIr, client_src: GlobalFusionClient<R>, client_dst: GlobalFusionClient<R>, stream: StreamId, ) -> FusionTensor<R>

where B: FusionBackend<FusionRuntime = R>,

Change the client of the given int tensor.

pub fn change_client_bool<B>( tensor: TensorIr, client_src: GlobalFusionClient<R>, client_dst: GlobalFusionClient<R>, stream: StreamId, ) -> FusionTensor<R>

where B: FusionBackend<FusionRuntime = R>,

Change the client of the given bool tensor.

pub fn change_client_quantized<B>( tensor: TensorIr, client_src: GlobalFusionClient<R>, client_dst: GlobalFusionClient<R>, stream: StreamId, ) -> FusionTensor<R>

where B: FusionBackend<FusionRuntime = R>,

Change the client of the given quantized tensor.

pub fn resolve_tensor_float<B>( &self, tensor: FusionTensor<R>, ) -> <B as BackendTypes>::FloatTensorPrimitive

where B: FusionBackend<FusionRuntime = R>,

Resolve the given float tensor to a primitive tensor.

pub fn resolve_tensor_int<B>( &self, tensor: FusionTensor<R>, ) -> <B as BackendTypes>::IntTensorPrimitive

where B: FusionBackend<FusionRuntime = R>,

Resolve the given int tensor to a primitive tensor.

pub fn resolve_tensor_bool<B>( &self, tensor: FusionTensor<R>, ) -> <B as BackendTypes>::BoolTensorPrimitive

where B: FusionBackend<FusionRuntime = R>,

Resolve the given bool tensor to a primitive tensor.

pub fn sync_collective<B>(&self, device: &<B as BackendTypes>::Device)

where B: FusionBackend<FusionRuntime = R>,

Synchronize the collective operations.

pub fn ensure_collective_init<B>(&self, device_ids: Vec<DeviceId>)

where B: FusionBackend<FusionRuntime = R>,

Ensure that communication between the given devices is initialized. Initializing communication is generally blocking, so we make sure to flush those operations.

Trait Implementations

impl<R> Clone for GlobalFusionClient<R>

where R: FusionRuntime,

fn clone(&self) -> GlobalFusionClient<R>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.