You can set this environment variable in order to specify to which REANA cloud instance your client should connect. For example:
$ export REANA_SERVER_URL=https://reana.cern.chYou should specify valid access token for the REANA cloud instance you would like to use. For example:
$ export REANA_ACCESS_TOKEN=XXXXXXXThe token should be provided to you by the REANA cluster administrators.
You can set this environment variable in order to specify a concrete workflow
you would like to work on. (As an alternative to specifying --workflow
option in commands.) For example:
$ export REANA_WORKON=myfirstanalysiswill work on the latest run of your "myfirstanalysis" workflow.
Note that you can also specify a concrete run number:
$ export REANA_WORKON=myfirstanalysis.42which will permit to work on the third run of the "myfirstanalysis" workflow, for example to check out past input and output files.
You can list all your workflow runs and their statuses by doing:
$ reana-client listand set REANA_WORKON to the one you would like to work on.
Please see the :ref:`gettingstarted` section for a basic reana-client use
case scenario.
Uploading files or directories to the analysis workspace is simple as:
$ reana-client upload mydata.csv mycode.C mytmp
File mydata.csv was successfully uploaded.
File mycode.C was successfully uploaded.
File mytmp/myfiltereddata.csv was successfully uploaded.If you want to upload all input files defined in the reana.yaml of the analysis, you can just run:
$ reana-client upload
File mydata.csv was successfully uploaded.
File mycode.C was successfully uploaded.Directory structures are maintained, i.e. mytmp exists in the workspace.
Note that symbolic links are resolved at the moment of upload so that a hard copy of the link target is uploaded to the cloud storage workspace. The link is not maintained throughout the workflow execution.
The deletion of files contained in the analysis workspace is possible through
the remove command:
$ reana-client rm mydata.csv
File mydata.csv was successfully deleted.
25356 bytes freed up.If you want to delete more than one file at once it is possible to use globbing:
$ reana-client rm '**/*.csv'
File mydata.csv was successfully deleted.
File mytmp/myfiltereddata.csv was successfully deleted.
79736 bytes freed up.The movement of file(s) or folders within the analysis workspace is
possible through the mv command:
$ reana-client mv data/mydata.csv mydata.csv
File mydata.csv was successfully deleted.If you need to use secrets in your workflow you can add them with
the secrets-add command:
$ # You can upload secrets from literal strings:
$ reana-client secrets-add --env CERN_USER=johndoe
--env CERN_KEYTAB=.keytab
Secrets CERN_USER, CERN_KEYTAB were successfully uploaded.
$ # ...and from files:
$ reana-client secrets-add --file ~/.keytab
Secrets .keytab were successfully uploaded.
$ # ...you can also combine two options in one command:
$ reana-client secrets-add --env CERN_USER=johndoe
--env CERN_KEYTAB=.keytab
--file ~/.keytab
Secrets .keytab, CERN_USER, CERN_KEYTAB were successfully uploaded.
$ # Trying to add a secret that is already added
$ # will result in a warning and no action will be taken:
$ reana-client secrets-add --env CERN_USER=johndoe
One of the secrets already exists. No secrets were added.
$ # If you are sure that you want to overwrite it you can use
$ # the ``--overwrite`` option:
$ reana-client secrets-add --env CERN_USER=janedoe
--overwrite
Secrets CERN_USER were successfully uploaded.
$ # Note that the ``--overwrite`` option will aply to
$ # all of secrets passed next to it.The added secrets will be available in your workflow execution container either
as environment variables (see example CERN_USER above) or as
mounted files (see .keytab example above) in the /etc/reana/secrets/
directory.
To list all the secrets that you have uploaded you can use
the secrets-list command:
$ reana-client secrets-list
NAME TYPE
.keytab file
CERN_KEYTAB env
CERN_USER envIf you want to delete some of the secrets that you have uploaded you can use
the secrets-delete command:
$ reana-client secrets-delete CERN_USER, CERN_KEYTAB
Secrets CERN_USER, CERN_KEYTAB were successfully deleted.If you want to run a workflow with different input parameters than the ones in
reana.yaml, you can do it by running reana-client start with flag -p
and specifying parameters that you want to override.
Note that parameters passed with -p flag must exist in reana.yaml.
Non-existing parameters will be skipped.
$ reana-client start -p myparam1=myval1 -p myparam2=myval2
myanalysis.42 has been started.If you have fully working analysis with reana.yaml, you can run the workflow
easily via the run wrapper command, which will create a new workflow, upload
analysis inputs, and start the workflow run.
$ vim reana.yaml
$ reana-client run -n myanalysis
[INFO] Creating a workflow...
myanalysis.42
[INFO] Uploading files...
File code/helloworld.py was successfully uploaded.
File data/names.txt was successfully uploaded.
[INFO] Starting workflow...
myanalysis.42 has been started.
$ export REANA_WORKON=myanalysis
$ reana-client status
NAME RUN_NUMBER CREATED STATUS PROGRESS
myanalysis 1 2018-11-07T12:45:18 running 1/1
$ reana-client download results/plot.pngSerial
Serial workflows can be executed partially until the step specified by the
user. To do so, you need to provide the target step name as an operational
option to the reana-client start or reana-client run commands.
$ reana-client start -w myanalysis.42 -o TARGET=gendata
# or
$ reana-client run -w myanalysis.42 -o TARGET=gendataIf you want to start workflow execution from a certain step you should use
operational option FROM with a desired step name.
$ reana-client start -w myanalysis.42 -o FROM=fitdata
# or
$ reana-client run -w myanalysis.42 -o FROM=fitdataIf you want to restart a workflow on the same workspace you should use a
restart command.
Note that workflow restarting can be used in a combination with operational
options FROM and TARGET. You can also pass a modified workflow
specification with -f or --file` flag.
# Restarts workflow on the same workspace from step named fitdata
$ reana-client restart -w myanalysis.42 -o FROM=fitdata
# Restarts workflow on the same workspace and starts execution from mystep3 to mystep7
$ reana-client restart -w myanalysis.42 -o FROM=mystep3 -o TARGET=mystep7
# Restarts workflow on the same workspace with a modified workflow specification
$ reana-client restart -w myanalysis.42 -f reana.yamlCWL
CWL allows executing workflows partially.
To do so, you need to provide the specific target as an operational option for
the reana-client start or reana-client run commands.
$ reana-client start -w myanalysis.42 -o --target=gendata
# or
$ reana-client run -w myanalysis.42 -o --target=gendataYadage
Not implemented yet.
Downloading files from an analysis workspace:
$ reana-client download result.png
File plot.png downloaded to /myfirstanalysis.In the same way you can download all outputs defined in the reana.yaml file of the analysis, by just running:
$ reana-client downloadNote that downloading directories is not yet supported.
While your analysis workflows are running, you may want to open interactive session processes on the workspace, such as Jupyter notebooks, via the open command. This will allow to quickly explore the generated data while the analysis is in progress, or even run your analyses from within the notebook environment spawned on the remote containerised platform.
$ reana-client open -w myanalysis.42 jupyter
https://reana.cern.ch/7cd4d23e-48d1-4f7f-8a3c-3a6d256fb8bc?token=P-IkL_7w25IDHhes8I7DtICWLNQm2WAZ9gkoKC2vq10
It could take several minutes to start the interactive session.Open the link returned by the command in order to access the interactive notebook session.
REANA currently supports Jupyter notebooks. Note that you can pass any notebook image you are interested to run on the workspace, such as PySpark, or even your own image, by using the --image option.
Once you finished working on your interactive session notebook, you can close it
via close command.
$ reana-client close -w myanalysis.42
Interactive session for workflow myanalysis.42 was successfully closedYou can mark a workflow as deleted with:
$ reana-client deletePassing no argument will mark the workflow selected by your REANA_WORKON variable as deleted. To specify a different workflow than your currently selected one use the -w/--workflow flag and set the workflow name and run number. If e.g. workflow run number 123 of your analysis failed, you can delete it as follows:
$ reana-client delete --workflow=myanalysis.42After simple deletion the workspace can be accessed to retrieve files uploaded there. If you are sure the workspace can also be deleted pass the --include-workspace flag.
$ reana-client delete --workflow=myanalysis.42 --include-workspaceTo delete all runs of a given workflow, pass the --include-all-runs flag and run:
$ reana-client delete --workflow=myanalysis --include-all-runsand to completely remove a workflow run and its workspace from REANA pass the --include-records flag:
$ reana-client delete --workflow=myanalysis.42 --include-recordsYou can stop a workflow with:
$ reana-client stop --forceThe workflow assigned to REANA_WORKON variable will be stopped. To specify a different workflow than your currently selected one use the -w/--workflow flag and set the workflow name or UUID.
$ reana-client stop --force --workflow=otherworkflow.1Note that currently only force stop is implemented.
You can get inspiration on how to structure your REANA-compatible research data
analysis from several reana-demo-... examples
provided on GitHub.
The full list of reana-client commands with their documented options is
available in the :ref:`cliapi` documentation.