PipelineIo¶
- class toasty.pipeline.PipelineIo[source]¶
Bases:
ABC
An abstract base class for I/O relating to pipeline processing. An instance of this class might be used to fetch files from, and send them to, a cloud storage system like S3 or Azure Storage.
Methods Summary
check_exists
(*path)Test whether an item at the specified path exists.
get_item
(*path[, dest])Fetch a file-like item at the specified path, writing its contents into the specified file-like object dest.
list_items
(*path)List the items contained in the folder at the specified path.
load_from_config
(path)Create a new I/O backend from saved configuration.
put_item
(*path[, source])Put a file-like item at the specified path, reading its contents from the specified file-like object source.
save_config
(path)Save this object's configuration to the specified filesystem path.
Methods Documentation
- abstract check_exists(*path)[source]¶
Test whether an item at the specified path exists.
- Parameters:
- *pathstrings
The path to the item, intepreted as components in a folder hierarchy.
- Returns:
- A boolean indicating whether the item in question exists.
- abstract get_item(*path, dest=None)[source]¶
Fetch a file-like item at the specified path, writing its contents into the specified file-like object dest.
- Parameters:
- *pathstrings
The path to the item, intepreted as components in a folder hierarchy.
- destwriteable file-like object
The object into which the item’s data will be written as bytes.
- Returns:
- None.
- abstract list_items(*path)[source]¶
List the items contained in the folder at the specified path.
- Parameters:
- *pathstrings
The path to the item, intepreted as components in a folder hierarchy.
- Returns:
- An iterable of
(stem, is_folder)
, where stem is the “basename” of an - item contained within the specified folder and is_folder is a boolean
- indicating whether this item appears to be a folder itself.
- An iterable of
- classmethod load_from_config(path)[source]¶
Create a new I/O backend from saved configuration.
- Parameters:
- pathpath-like
The path where the configuration was saved.
- Returns:
- A new instance implementing the PipelineIO abstract base class.
- abstract put_item(*path, source=None)[source]¶
Put a file-like item at the specified path, reading its contents from the specified file-like object source.
- Parameters:
- *pathstrings
The path to the item, intepreted as components in a folder hierarchy.
- sourcereadable file-like object
The object from which the item’s data will be read, as bytes.
- Returns:
- None.