toasty pipeline init
¶
The init
pipeline command creates a pipeline workspace.
Usage¶
toasty pipeline init
[--azure-conn-env VARNAME]
[--azure-container CONTAINERNAME]
[--azure-path-prefix PREFIX]
[--local PATH]
[WORKDIR]
The WORKDIR
argument is the name of a directory that will be created as the
pipeline workspace.
The --azure-*
arguments configure the workspace to connect to an Azure
storage account for publishing data. The --local
argument indicates that
“publishing” should happen by copying files elsewhere on the local filesystem.
One of these two storage options must be activated.
Examples¶
To set up a workspace connected to Azure storage:
toasty pipeline init \
--azure-conn-env=AZURE_STORAGE_CONNECTION_STRING \
--azure-container=feeds \
--azure-path-prefix=noirlab \
workspace
After running this command, a new directory workspace
will be created.
Note that the --azure-conn-env
argument takes the name of an environment
variable. The value of that environment variable should then contain an
Azure storage “connection string”. This indirect approach avoids the security
issues that would happen if you just passed the the connection string directly
to the program.
To set up a workspace using local storage (likely for testing):
toasty pipeline init \
--local=/tmp/pipetest \
testspace
In order for subsequent commands to work, you will need to place a
toasty-pipeline-config.yaml
file in the local directory (/tmp/pipetest
in this example).
Notes¶
After initialization, the named WORKDIR
directory will be created and will
contain a file name toasty-store-config.yaml
that records the storage
configuration options used when this command is called. The next step is to
invoke the toasty pipeline refresh command to query for candidate imags to
process.