Skip to main content

DataLoader

Trait DataLoader 

pub trait DataLoader<O>: Send + Sync {
    // Required methods
    fn iter<'a>(
        &'a self,
    ) -> Box<dyn DataLoaderIterator<O, Item = Result<O, DatasetError>> + 'a>;
    fn num_items(&self) -> usize;
    fn to_device(&self, device: &Device) -> Arc<dyn DataLoader<O>> ⓘ;
    fn slice(&self, start: usize, end: usize) -> Arc<dyn DataLoader<O>> ⓘ;
}
Expand description

A data loader that can be used to iterate over a dataset.

Required Methods§

fn iter<'a>( &'a self, ) -> Box<dyn DataLoaderIterator<O, Item = Result<O, DatasetError>> + 'a>

Returns a boxed iterator to iterate over the data loader.

fn num_items(&self) -> usize

The number of items (not the number of batches nor the number of iterations), corresponding to the items_total of the progress returned by the iterator.

fn to_device(&self, device: &Device) -> Arc<dyn DataLoader<O>> ⓘ

Move the data loader to the given device, ensuring the batches are assigned to the correct device.

fn slice(&self, start: usize, end: usize) -> Arc<dyn DataLoader<O>> ⓘ

Returns a new data loader containing a subset of the data.

The subset includes items from start (inclusive) to end (exclusive), preserving the batch size and ordering of the original data loader.

§Arguments
  • start - The starting index of the subset (inclusive).
  • end - The ending index of the subset (exclusive).
§Returns

A boxed DataLoader instance containing only the specified range.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§

§

impl<I, O> DataLoader<O> for BatchDataLoader<I, O>
where I: Send + Sync + Clone + 'static, O: Send + 'static,

§

impl<I, O> DataLoader<O> for MultiThreadDataLoader<I, O>
where I: Send + Sync + Clone + 'static, O: Send + 'static + Debug,