Skip to content

Latest commit

 

History

History
129 lines (99 loc) · 5.3 KB

deepvariant-gcp-info.md

File metadata and controls

129 lines (99 loc) · 5.3 KB
Raw

Getting Started with GCP

DeepVariant doesn't require GCP, but if you want to use it, these are some instructions that we found to be useful when getting started.

Set up a Google Cloud account

To get started using DeepVariant on Google Cloud Platform (GCP), you first need to set up an account and a project to contain your cloud resources.

  • If you do not have an account yet, you should create one at cloud.google.com. You should then enable billing for your account but note that if your account is new, you receive $300 of free credit. Once your cloud account is set up, you should be able to log in to the Cloud Console to view or administer your cloud resources.

  • From the Cloud Console, set up a project to house all of the cloud resources (storage, compute, services) that you will associate with your use of DeepVariant. For example, if your organization is AcmeCorp, you might call your project acmecorp-deepvariant.

  • Finally, please visit the "Compute Engine" page on Cloud Console. You don't need to create Compute Engine instances at this time, but simply visiting this page will initialize your compute engine "service account" so that we can authorize it.

(As you progress in your use of Google Cloud Platform, you will likely find it useful to create a Cloud Organization to house your projects. Here are some best practices for organizating cloud projects for an enterprise.)

Install the Google Cloud SDK

The Google Cloud SDK comes with two very useful command line utilities that you can use on your local workstation---gcloud, which lets you administer your cloud resources, and gsutil, which lets you manage and transfer data to Google Cloud Storage buckets. We will make use of these tools in the following instructions. To install the Cloud SDK, follow the installation instructions here.

The final step in the installation process (gcloud init) will have you authenticate via your web browser and select a default zone and region for your cloud resources, which you can choose based on your location and regional hardware availability.

NOTE: Not all zones are equipped with GPUs, so if you want to use GPUs for your project, please take note of the availability listing here.

To verify that the installation and authentication succeeded, run

gcloud auth list

and verify that your account email address is printed.

Starting a Compute Engine instance

A simple way to access compute on GCP is Google Compute Engine. Compute Engine instances can be sized to meet computational and storage needs for your project.

Before we get started, ensure you have adequate quota provisioned so that you can get all the CPUs/GPUs that you need. To start with, you might want to request quota for 64 CPUs and 2 GPUs in your zone.

DeepVariant can make use of multiple CPU cores and (currently, a single) GPU device. For this "quick start" guide, let's allocate an 8-core non-preemptible instance in your default zone with a single GPU, running Ubuntu 16.04, with a disk of reasonable size for modest work with genomic data. From our local command line, we do:

gcloud beta compute instances create "${USER}-deepvariant-quickstart" \
  --scopes "compute-rw,storage-full,cloud-platform"  \
  --image-family ubuntu-1604-lts --image-project ubuntu-os-cloud \
  --machine-type n1-standard-8  \
  --boot-disk-size=200GB \
  --zone us-west1-b \
  --accelerator type=nvidia-tesla-k80,count=1 --maintenance-policy TERMINATE --restart-on-failure

NOTE: To create an instance without GPU, simply omit the last line from the command.

Check that the instance has been created and started:

gcloud compute instances list

which should produce output like:

NAME                    ZONE        MACHINE_TYPE    PREEMPTIBLE   INTERNAL_IP  EXTERNAL_IP     STATUS
[USER]-deepvariant-quickstart  us-west1-b  n1-standard-8                 10.138.0.4   35.185.203.59   RUNNING

Then connect to your instance via SSH:

gcloud compute ssh --zone us-west1-b "${USER}-deepvariant-quickstart"

You should land at a shell prompt in your new instance!

NOTE: All of these steps can also be completed from the Cloud Console, if you prefer. Consult this guide, but be sure to choose Ubuntu 16.04 as your image, as DeepVariant has not been tested on other Linux distributions.

For more information about getting started with Compute Engine, see: