Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [GCP Batch] Support passing standard machine types to the Google backend #1

Merged
merged 2 commits into from
Oct 10, 2024

Conversation

javiergaitan
Copy link

@javiergaitan javiergaitan commented Oct 4, 2024

Context

This change rebases broadinstitute#7545 as well as broadinstitute#7518 on top of the deepgenomics/dg-ci branch. The goal is to port only the functionality necessary to support arbitrary machine types in wdl workflows into our fork of the cromwell repository.

Changes included

Testing

  1. Ran unit and integration tests
# Start sbt console
sbt
# Only relevant tests
testOnly cromwell.backend.google.batch.util.GcpBatchMachineConstraintsSpec
# All tests
test
  1. Deployed and ran a wdl workflow to validate the change
version 1.0

task nvidia_smi {
    input {
        String docker_version
    }

    command <<<
        nvidia-smi

        touch .done
        echo "Finished at $(date)"
    >>>

    runtime {
        docker: <internal image>
        disks: "local-disk 50 SSD"
        memory: "32G"
        preemptible: 0
        gpuCount: 1
        gpuType: "nvidia-tesla-a100"
        standardMachineType: "a2-highgpu-1g"
    }

    output {
        File done = ".done"
    }
}

workflow nvidia_smi_wf {
    input {
        String docker_version
    }
    
    call nvidia_smi  as nvidia_smi_call {
        input:
            docker_version = docker_version
    }

    output {
        File done = ".done"
    }
}
  1. Verified the machine type on GCP batch as A2 and the output of nvidia-smi showing the requested A100 accelerator.

@ovesh ovesh merged commit ae4e6aa into dg-ci Oct 10, 2024
5 of 13 checks passed
@ovesh ovesh deleted the support-standard-machine-type-google-backend-rebased branch October 10, 2024 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants