Feat: release 2.8.0 - adds hpc support

[dapineyro]

Overview

This PR adds preliminary support for HPC execution platform in CloudOS. It will allow to submit HPC jobs in workspaces in which it's prepared to.

Jira ticket

LP-9412
LIF-166

Changes

• Adds a new option for --execution-platform parameter, now accepting hpc.
• Adds a new parameter --hpc-id to provide the ID. Note that the default value included is only added for internal testing purposes.
• If --execution-platform hpc is used, a new message is displayed in stdout to tell the user some parameters of cloudos-cli are not valid and will be ignored.
• Modify the job submission call to adapt to HPC executor.
• Bumps version to 2.8.0, updating changelog.md, readme.md, _version.py and pytests to support the new parameter.
• Created a new docker image v2.8.0.

Tests (on DEV)

Running docker image:

docker run --rm -it quay.io/lifebitaiorg/cloudos-cli:v2.8.0

Setting up variables:

MY_API_KEY="xxxx"
CLOUDOS="https://dev.sdlc.lifebit.ai"
WORKSPACE_ID="xxxx"
PROJECT_NAME="davidp_testing"
WORKFLOW_NAME="Cufflinks pipeline"
JOB_PARAMS="/cloudos/examples/rnatoy.config"

CloudOS-CLI job using the new HPC execution platform

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name "$WORKFLOW_NAME" \
    --job-config $JOB_PARAMS \
    --execution-platform hpc \
    --job-name "test_1_hpc_cloudos_cli"

CloudOS python package: a package for interacting with CloudOS.

Version: 2.8.0

CloudOS job functionality: run and check jobs in CloudOS.

Executing run...

HPC execution platform selected
[Message] Please, take into account that HPC execution do not support the following parameters and all of them will be ignored:
	--resumable
	--job-queue
	--instance-type | --instance-disk | --spot | --cost-limit
	--storage-mode | --lustre-size
	--wdl-mainfile | --wdl-importsfile | --cromwell-token

	Job successfully launched to CloudOS, please check the following link: https://dev.sdlc.lifebit.ai/app/jobs/6610210f1fe8bb67eeb11ba9
	Your assigned job id is: 6610210f1fe8bb67eeb11ba9

	Your current job status is: running
	To further check your job status you can either go to https://dev.sdlc.lifebit.ai/app/jobs/6610210f1fe8bb67eeb11ba9 or use the following command:
	cloudos job status \
		--apikey $MY_API_KEY \
		--cloudos-url https://dev.sdlc.lifebit.ai \
		--job-id 6610210f1fe8bb67eeb11ba9

CloudOS-CLI job using the new HPC execution platform, but using useless parameters

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name "$WORKFLOW_NAME" \
    --job-config $JOB_PARAMS \
    --execution-platform hpc \
    --resumable --job-queue "non-existing-queue" --spot --cost-limit 10 \
    --instance-type "c5.xlarge" --instance-disk 40 --storage-mode lustre --lustre-size 1200 \
    --wdl-mainfile "main.wdl" --wdl-importsfile "imports.wdl" --cromwell-token aaa \
    --job-name "test_2_hpc_useless_params_cloudos_cli"

CloudOS python package: a package for interacting with CloudOS.

Version: 2.8.0

CloudOS job functionality: run and check jobs in CloudOS.

Executing run...

HPC execution platform selected
[Message] Please, take into account that HPC execution do not support the following parameters and all of them will be ignored:
	--resumable
	--job-queue
	--instance-type | --instance-disk | --spot | --cost-limit
	--storage-mode | --lustre-size
	--wdl-mainfile | --wdl-importsfile | --cromwell-token

	Job successfully launched to CloudOS, please check the following link: https://dev.sdlc.lifebit.ai/app/jobs/661020af1fe8bb67eeb0f74d
	Your assigned job id is: 661020af1fe8bb67eeb0f74d

	Your current job status is: running
	To further check your job status you can either go to https://dev.sdlc.lifebit.ai/app/jobs/661020af1fe8bb67eeb0f74d or use the following command:
	cloudos job status \
		--apikey $MY_API_KEY \
		--cloudos-url https://dev.sdlc.lifebit.ai \
		--job-id 661020af1fe8bb67eeb0f74d

CloudOS-CLI job using default AWSbatch

cloudos job run \
    --cloudos-url $CLOUDOS \
    --apikey $MY_API_KEY \
    --workspace-id $WORKSPACE_ID \
    --project-name "$PROJECT_NAME" \
    --workflow-name "$WORKFLOW_NAME" \
    --job-config $JOB_PARAMS \
    --job-name "test_3_aws_cloudos_cli"

CloudOS python package: a package for interacting with CloudOS.

Version: 2.8.0

CloudOS job functionality: run and check jobs in CloudOS.

Executing run...
	No job_queue was specified, using the CloudOS default queue: E2E_20240405_cwoa1.
	Job successfully launched to CloudOS, please check the following link: https://dev.sdlc.lifebit.ai/app/jobs/6610221a1fe8bb67eeb1659b
	Your assigned job id is: 6610221a1fe8bb67eeb1659b

	Your current job status is: initializing
	To further check your job status you can either go to https://dev.sdlc.lifebit.ai/app/jobs/6610221a1fe8bb67eeb1659b or use the following command:
	cloudos job status \
		--apikey $MY_API_KEY \
		--cloudos-url https://dev.sdlc.lifebit.ai \
		--job-id 6610221a1fe8bb67eeb1659b

[danielboloc]

LGTM! Very nice addition!

I just added some tiny format suggestions

[dapineyro]

LGTM! Very nice addition!

I just added some tiny format suggestions

Thanks for the suggestions @danielboloc ! I've accepted all of them 👍