Switch to Artifact Registry #29
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vladsavelyev
force-pushed
the
artefact-registry
branch
from
40bf022 to
2ed804e
Compare
lgruen
suggested changes
Feb 15, 2021
vladsavelyev
force-pushed
the
artefact-registry
branch
from
0d4c1e0 to
cdb8c94
Compare
… for AR, user AR for images
vladsavelyev
force-pushed
the
artefact-registry
branch
from
de9fceb to
e80809f
Compare
This was referenced Feb 17, 2021
vladsavelyev
added a commit
that referenced
this pull request
Feb 25, 2021
* config.mk: use AR docker image * build.yaml: use AR for images * Gitignore artifacts * fetch-image.sh: gcloud auth for AR * image-fetcher dockerfile: base on AR * image fetcher make file: use AR * build-batch-worker-image-startup.sh: add gcloud auth configure-docker for AR, user AR for images * batch worker: ue AR for public_gcr_images * Worker dockerfile: use AR * ci build.py: add gcloud auth configure-docker for AR, use AR base image * Makefiles: use AR docker images * stat-permissions.sh: fix for macOS gnu-grep * third-party dockers: use AR * ci dockerFile create_namespace: use AR image * Docker hail-ubuntu: use AR image * ci build.py: add region into the jinja render config for Dockerfiles * CI deploy config: add region * Python-dill: use AR image * Auth deployment.yaml: add REGION * Use hail-dvc default ci image pointer * Add Artifact Registry in main.tf * Add AR in dev-docs * Add AR for ukbb-rg * Add AR in more makefiles * Fix AR usage for image-fetcher * Specify "region" in the "hail-ubuntu-stmp" target. * Fix zone assignment in startup script. * Allow ci to read from Artifact Registry. * Makefiles: parametrise `DOCKER_ROOT_IMAGE` to support both GCR and AR * Parametrise `DOCKER_PREFIX` to support both GCR and AR * Un-gitignore "vlad" * Fix * batch: pass DOCKER_PREFIX to worker * DOCKER_PREFIX in build-batch-worker-image-startup.sh * build.yaml: add ubuntu tags back * ci/ci/build.py: parametrise docker prefix * infra README: configure-docker AR Co-authored-by: Leonhard Gruenschloss <leonhard.gruenschloss@populationgenomics.org.au>
vladsavelyev
added a commit
that referenced
this pull request
Apr 27, 2021
…#10347) * Switch to Artifact Registry (#29) * config.mk: use AR docker image * build.yaml: use AR for images * Gitignore artifacts * fetch-image.sh: gcloud auth for AR * image-fetcher dockerfile: base on AR * image fetcher make file: use AR * build-batch-worker-image-startup.sh: add gcloud auth configure-docker for AR, user AR for images * batch worker: ue AR for public_gcr_images * Worker dockerfile: use AR * ci build.py: add gcloud auth configure-docker for AR, use AR base image * Makefiles: use AR docker images * stat-permissions.sh: fix for macOS gnu-grep * third-party dockers: use AR * ci dockerFile create_namespace: use AR image * Docker hail-ubuntu: use AR image * ci build.py: add region into the jinja render config for Dockerfiles * CI deploy config: add region * Python-dill: use AR image * Auth deployment.yaml: add REGION * Use hail-dvc default ci image pointer * Add Artifact Registry in main.tf * Add AR in dev-docs * Add AR for ukbb-rg * Add AR in more makefiles * Fix AR usage for image-fetcher * Specify "region" in the "hail-ubuntu-stmp" target. * Fix zone assignment in startup script. * Allow ci to read from Artifact Registry. * Makefiles: parametrise `DOCKER_ROOT_IMAGE` to support both GCR and AR * Parametrise `DOCKER_PREFIX` to support both GCR and AR * Un-gitignore "vlad" * Fix * batch: pass DOCKER_PREFIX to worker * DOCKER_PREFIX in build-batch-worker-image-startup.sh * build.yaml: add ubuntu tags back * ci/ci/build.py: parametrise docker prefix * infra README: configure-docker AR Co-authored-by: Leonhard Gruenschloss <leonhard.gruenschloss@populationgenomics.org.au> * Fix building worker image (#40) * Add Artifact Registry permissions for the batch-agent service account. * Gitignore */hail.jar and *.dylib * Comment in gitignore on dylib * Default use_artifact_registry to false * Use DOCKER_ROOT_IMAGE to pass to batch worker image * Use DOCKER_ROOT_IMAGE wherever makes sense * Render build-batch-worker-image-startup.sh with Jinja to set up docker root image * Fix jinja params * Add phony rule to configure gcloud and Docker * Add comments on HAIL_DOCKER_PREFIX * Make sure the env variable HAIL_DOCKER_PREFIX available to the batch worker instance * Add missing docker_root_image * Infra README: correct env var name * Fix env var case * remove other instances of gcr.io in favor of DOCKER_PREFIX * add hail_version to gitignore once and for all * debug * add back yaml changes after rebase * remove gcloud target from hail/Makefile Co-authored-by: Vlad Savelyev <vladislav.sav@gmail.com> Co-authored-by: Leonhard Gruenschloss <leonhard.gruenschloss@populationgenomics.org.au> Co-authored-by: vladsaveliev <vladislav.savelyev@populationgenomics.org.au>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Switch pointers from GCR to Artifact Registry (GCR is not available in Australia).
Also, a tiny fix to allow using GNU tools on macOS to get file permissions, and gitignored macOS dylib.
UPD: downstream fixes after merging: #40, #41