Pyramid

class toasty.pyramid.Pyramid[source]

Bases: object

An object representing a tile pyramid.

Pyramids may be “generic”, without any associated world coordinate information, or TOAST-based, where each tile in the pyramid has associated spatial information. TOAST-based pyramids may have a spatial filter applied if only a subset of tiles are known to be of interest. In both cases, the pyramid may be limited to a “subpyramid”.

The key interface of this class is the walk() method, which provides a generic way to visit locations in the pyramid with flexible filtering and the possibility of performing the walk at a high level of parallelization.

Attributes Summary

depth

The maximum depth of the pyramid, a nonnegative integer.

Methods Summary

count_leaf_tiles()

Count the number of leaf tiles in the current pyramid.

count_live_tiles()

Count the number of "live" tiles in the current pyramid.

count_operations()

Count the number of operations needed to perform a reduction over the pyramid's live tiles.

new_generic(depth)

Define a tile pyramid without TOAST coordinate information.

new_toast(depth[, coordsys])

Define a TOAST tile pyramid.

new_toast_filtered(depth, tile_filter[, ...])

Define a TOAST tile pyramid where some tiles are filtered out.

subpyramid(apex)

Configure this pyramid to only visit a sub-pyramid.

visit_leaves(callback[, parallel, cli_progress])

Traverse the pyramid, calling the callback for each leaf tile.

walk(callback[, parallel, cli_progress])

Walk the pyramid depth-first, calling the callback for each live tile.

Attributes Documentation

depth = None

The maximum depth of the pyramid, a nonnegative integer. This value may be changed.

Methods Documentation

count_leaf_tiles()[source]

Count the number of leaf tiles in the current pyramid.

Returns:
The number of leaf tiles.

Notes

A leaf tile is a tile whose level is equal to the depth of the pyramid.

This calculation is non-trivial if tile filtering is in effect, and in fact requires iterating over essentially the entire pyramid to ensure that all of the tiles at the deepest layer are visited.

count_live_tiles()[source]

Count the number of “live” tiles in the current pyramid.

Returns:
The number of live tiles.

Notes

A tile is “live” if it is a leaf tile matching the tile filter and/or subpyramid limitation (if active), or if any of its children are live. Tiles above the subpyramid apex, if a subpyramid limitation is active, are not live. A leaf tile is a tile whose level is equal to the depth of the pyramid.

count_operations()[source]

Count the number of operations needed to perform a reduction over the pyramid’s live tiles.

Returns:
The number of needed operations.

Notes

See count_live_tiles() for a definition of liveness. The return value of this function is equal to the number of live tiles that are not also leaf tiles. Therefore, count_operations() + count_leaf_tiles() = count_live_tiles().

classmethod new_generic(depth)[source]

Define a tile pyramid without TOAST coordinate information.

Parameters:
depthint

The tile depth to recurse to.

Returns:
New Pyramid instance.
classmethod new_toast(depth, coordsys=None)[source]

Define a TOAST tile pyramid.

Parameters:
depthint

The tile depth to recurse to.

coordsysoptional ToastCoordinateSystem

The TOAST coordinate system to use. Default is ASTRONOMICAL.

Returns:
New Pyramid instance.
classmethod new_toast_filtered(depth, tile_filter, coordsys=None)[source]

Define a TOAST tile pyramid where some tiles are filtered out.

Parameters:
depthint

The tile depth to recurse to.

tile_filterfunction(Tile)->bool

A tile filter function; only tiles for which the function returns True will be investigated.

coordsysoptional ToastCoordinateSystem

The TOAST coordinate system to use. Default is ASTRONOMICAL.

Returns:
New Pyramid instance.
subpyramid(apex)[source]

Configure this pyramid to only visit a sub-pyramid.

Parameters:
apexPos

The apex of the sub-pyramid to visit.

Returns:
Self.

Notes

After calling this function, iterations over this pyramid will only return tiles below and including apex. The depth of apex may not be larger than the pyramid’s total depth.

It is not legal to call this function more than once on the same Pyramid instance.

visit_leaves(callback, parallel=None, cli_progress=False)[source]

Traverse the pyramid, calling the callback for each leaf tile.

Parameters:
callbackfunction(Pos, Optional[Tile])
-> None

A function to be called for all of the leaves.

parallelinteger or None (the default)

The level of parallelization to use. If unspecified, defaults to using all CPUs. If the OS does not support fork-based multiprocessing, parallel processing is not possible and serial processing will be forced. Pass 1 to force serial processing.

cli_progressoptional boolean, defaults False

If true, a progress bar will be printed to the terminal.

Returns:
None.

Notes

Use this function to visit all of the leaves of the pyramid, with the possibility of parallelizing the computation substantially using Python’s multiprocessing framework.

Here, “all” of the leaves means ones that have not been filtered out by the use of a TOAST tile filter and/or subpyramid selection.

The callback is passed the position of the leaf tile and, if the pyramid has been defined as a TOAST tile pyramid, its TOAST coordinate information. If not, the second argument to the callback will be None. In the corner case that this pyramid has depth 0, the TOAST tile information will be None even if this is a TOAST pyramid, because the TOAST tile information is not well-defined at level 0.

In the parallelized case, callbacks may occur in different processes and so cannot communicate with each other in memory, nor can they communicate with the parent process (that is, the process in which this function was invoked). No guarantees are made about the order in which leaf tiles are visited.

walk(callback, parallel=None, cli_progress=False)[source]

Walk the pyramid depth-first, calling the callback for each live tile.

Parameters:
callbackfunction(Pos) -> None

A function to be called across the pyramid.

parallelinteger or None (the default)

The level of parallelization to use. If unspecified, defaults to using all CPUs. If the OS does not support fork-based multiprocessing, parallel processing is not possible and serial processing will be forced. Pass 1 to force serial processing.

cli_progressoptional boolean, defaults False

If true, a progress bar will be printed to the terminal.

Returns:
None.

Notes

Use this function to perform a calculation over each tile in the pyramid, visiting in a depth-first fashion, with the possibility of parallelizing the computation substantially using Python’s multiprocessing framework.

In order to allow for efficient parallelization, there are important limitations on the what can occur inside the callback. It is guaranteed that when the callback for a given position is called, the callback has already been called for that position’s children. The callback may not have been called for all of those children, however, if tile filtering is in effect.

Callbacks may occur in different processes and so cannot communicate with each other in memory. If you need to transfer information between different callbacks, you must use a PyramidIO instance, write data to disk, and read the data later. For calculations whose intermediate products don’t merit long-term storage, consider using a temporary pyramid.

The callback is not called for the “leaf” positions in the pyramid, which are the ones at the pyramid’s depth level.