Struct SqliteDatasetWriter

pub struct SqliteDatasetWriter<I> { /* private fields */ }

This SqliteDatasetWriter struct is a SQLite database writer dedicated to storing datasets. It retains the current writer’s state and its database connection.

Being thread-safe, this writer can be concurrently used across multiple threads.

Typical applications include:

• Generation of a new dataset
• Storage of preprocessed data or metadata
• Enlargement of a dataset’s item count post preprocessing

Implementations

impl<I> SqliteDatasetWriter<I>

where I: Clone + Send + Sync + Serialize + DeserializeOwned,

pub fn new<P>( db_file: P, overwrite: bool, ) -> Result<SqliteDatasetWriter<I>, SqliteDatasetError>

where P: AsRef<Path>,

Creates a new instance of SqliteDatasetWriter.

Arguments

• db_file - A reference to the Path that represents the database file path.
• overwrite - A boolean indicating if the existing database file should be overwritten.

Returns

• A Result which is Ok if the writer could be created, Err otherwise.

pub fn write(&self, split: &str, item: &I) -> Result<usize, SqliteDatasetError>

Serializes and writes an item to the database. The item is written to the table for the specified split. If the table does not exist, it is created. If the table exists, the item is appended to the table. The serialization is done using the MessagePack

Arguments

• split - A string slice that defines the data split for writing (e.g., “train”, “test”).
• item - A reference to the item to be written to the database.

Returns

• A Result containing the index of the inserted row if successful, an error otherwise.

pub fn set_completed(&mut self) -> Result<(), SqliteDatasetError>

Marks the dataset as completed and persists the temporary database file.

Trait Implementations

impl<I> Debug for SqliteDatasetWriter<I>

where I: Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.