Feat: release 2.9.0 - adds workflow import

[dapineyro]

Overview

This PR implements the first iteration for the workflow import support.

Jira ticket

LIF-171

Changes

• Adds new command cloudos workflow import to import Nextflow pipelines to cloudos.
• The new command adds the following new parameters:
  • • --workflow-url: it expects the repo URL.
  • • --workflow-name: the new name for the workflow in CloudOS.
  • • --repository-project-id: ID of the repository owner or project.
  • • --repository-id: ID of the repo.
• If the user is trying to use this feature in a non-prepared workspace, an explanatory error message will be shown.
• Updates CHANGELOG.md, README.md and _version.py
• Bumps version to v2.9.0
• Adds a new Pytest for testing the new command
• New docker image available (v2.9.0)

NOTES about limitations in this implementation

This implementation can be considered iteration 1. It has the following limitations:

• The user has to collect --repository-project-id and --repository-id. In future iterations, this process will be done automatically.
• If the user adds a mistake in any of the values, the pipeline will be created anyway. CloudOS is not doing any check for the values for being correct. That will be solved in future iterations.
• It only supports Nextflow workflows, and the main File should be named main.nf.

Tests

The following tests are performed on https://dev.sdlc.lifebit.ai, which is the only CloudOS with the feature enable as of today.

Running docker image:

docker run --rm -it quay.io/lifebitaiorg/cloudos-cli:v2.9.0

Calling the in-built help:

cloudos workflow import --help
CloudOS python package: a package for interacting with CloudOS.

Version: 2.9.0

CloudOS workflow functionality: list and import workflows.

Usage: cloudos workflow import [OPTIONS]

  Imports workflows to CloudOS.

Options:
  -k, --apikey TEXT               Your CloudOS API key  [required]
  -c, --cloudos-url TEXT          The CloudOS url you are trying to access to.
                                  Default=https://cloudos.lifebit.ai.
  --workspace-id TEXT             The specific CloudOS workspace id.
                                  [required]
  --workflow-url TEXT             URL of the workflow to import. Please, note
                                  that it should be the URL shown in the
                                  browser, and it should come without any of
                                  the .git or /browse extensions.  [required]
  --workflow-name TEXT            The name that the workflow will have in
                                  CloudOS  [required]
  --repository-project-id INTEGER
                                  The ID of your repository project
                                  [required]
  --repository-id INTEGER         The ID of your repository. Only required for
                                  GitHub repositories
  --disable-ssl-verification      Disable SSL certificate verification.
                                  Please, remember that this option is not
                                  generally recommended for security reasons.
  --ssl-cert TEXT                 Path to your SSL certificate file.
  --help                          Show this message and exit.

Setting up variables:

MY_API_KEY="xxxx"
CLOUDOS="https://dev.sdlc.lifebit.ai"
WORKSPACE_ID="xxxx"

Import a GitHub pipeline

 cloudos workflow import \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --workflow-url $WORKFLOW_URL \
    --workflow-name "test_import_github_4" \
    --repository-project-id $REPOSITORY_PROJECT_ID \
    --repository-id $REPOSITORY_ID

CloudOS python package: a package for interacting with CloudOS.

Version: 2.9.0

CloudOS workflow functionality: list and import workflows.

Executing workflow import...

	[Message] Only Nextflow workflows are currently supported.

	Workflow test_import_github_4 was imported successfully with the following ID: 6616b30a454b09bbb3dc99e4

Import a Bitbucket server pipeline

cloudos workflow import \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --workflow-url $WORKFLOW_URL \
    --workflow-name "test_import_bitbucket_3" \
    --repository-project-id $REPOSITORY_PROJECT_ID
CloudOS python package: a package for interacting with CloudOS.

Version: 2.9.0

CloudOS workflow functionality: list and import workflows.

Executing workflow import...

	[Message] Only Nextflow workflows are currently supported.

	Workflow test_import_bitbucket_3 was imported successfully with the following ID: 6616ae51454b09bbb3db9987

Trying to use import in a non-prepared workspace 🔴

 cloudos workflow import \
    --cloudos-url "https://cloudos.lifebit.ai" \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --workflow-url $WORKFLOW_URL \
    --workflow-name "test_import_github_4" \
    --repository-project-id $REPOSITORY_PROJECT_ID \
    --repository-id $REPOSITORY_ID

CloudOS python package: a package for interacting with CloudOS.

Version: 2.9.0

CloudOS workflow functionality: list and import workflows.

Executing workflow import...

	[Message] Only Nextflow workflows are currently supported.

Traceback (most recent call last):
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/bin/cloudos", line 8, in <module>
    sys.exit(run_cloudos_cli())
             ^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/.virtualenvs/cloudos-cli/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/Documents/Lifebit/Lifebit_github/cloudos-cli/cloudos/__main__.py", line 829, in import_workflows
    workflow_id = cl.workflow_import(workspace_id,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dpineyro/Documents/Lifebit/Lifebit_github/cloudos-cli/cloudos/clos.py", line 606, in workflow_import
    raise ValueError('It seems your API key is not authorised. Please check if ' +
ValueError: It seems your API key is not authorised. Please check if your workspace has support for importing workflows using cloudos-cli

[danielboloc]

Amazing addition @dapineyro, LGTM!