toasty pipeline init¶
The init pipeline command creates a pipeline workspace.
Usage¶
toasty pipeline init
[--azure-conn-env VARNAME]
[--azure-container CONTAINERNAME]
[--azure-path-prefix PREFIX]
[--local PATH]
[WORKDIR]
The WORKDIR argument is the name of a directory that will be created as the
pipeline workspace.
The --azure-* arguments configure the workspace to connect to an Azure
storage account for publishing data. The --local argument indicates that
“publishing” should happen by copying files elsewhere on the local filesystem.
One of these two storage options must be activated.
Examples¶
To set up a workspace connected to Azure storage:
toasty pipeline init \
--azure-conn-env=AZURE_STORAGE_CONNECTION_STRING \
--azure-container=feeds \
--azure-path-prefix=noirlab \
workspace
After running this command, a new directory workspace will be created.
Note that the --azure-conn-env argument takes the name of an environment
variable. The value of that environment variable should then contain an
Azure storage “connection string”. This indirect approach avoids the security
issues that would happen if you just passed the the connection string directly
to the program.
To set up a workspace using local storage (likely for testing):
toasty pipeline init \
--local=/tmp/pipetest \
testspace
In order for subsequent commands to work, you will need to place a
toasty-pipeline-config.yaml file in the local directory (/tmp/pipetest
in this example).
Notes¶
After initialization, the named WORKDIR directory will be created and will
contain a file name toasty-store-config.yaml that records the storage
configuration options used when this command is called. The next step is to
invoke the toasty pipeline refresh command to query for candidate imags to
process.
