Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overhaul runners, add GCP & K8s stubs #105

Merged
merged 10 commits into from
Sep 17, 2021
Merged

overhaul runners, add GCP & K8s stubs #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
8012b3a
overhaul self-hosted runners
casperdcl Sep 14, 2021
966bfdc
GCP & K8s stubs
casperdcl Sep 14, 2021
50857c9
better headings
casperdcl Sep 14, 2021
d6ebe12
AWS dual-purpose for cloud storage
casperdcl Sep 14, 2021
6c402df
update CLI options
casperdcl Sep 14, 2021
d8c98d6
update cross-references
casperdcl Sep 14, 2021
94a70c3
review comments
casperdcl Sep 17, 2021
ea28f1e
Compute Resource vs Storage Provider
casperdcl Sep 17, 2021
2e2ec61
fix container usage
casperdcl Sep 17, 2021
f89fd98
typomg
casperdcl Sep 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions content/docs/cml-with-dvc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
container: docker://iterativeai/cml:0-dvc2-base1
container: docker://ghcr.io/iterative/cml:0-dvc2-base1
steps:
- uses: actions/checkout@v2
- name: cml_run
Expand Down Expand Up @@ -58,8 +58,7 @@ jobs:
cml-send-comment report.md
```

If you're using DVC with cloud storage, take note of environmental variables for
your storage format.
## Cloud Storage Provider Credentials

<details>

Expand Down
286 changes: 179 additions & 107 deletions content/docs/self-hosted-runners.md
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content/docs/start/github.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ supported CI systems.
jobs:
run:
runs-on: [ubuntu-latest]
container: docker://iterativeai/cml:0-dvc2-base1
container: docker://ghcr.io/iterative/cml:0-dvc2-base1
steps:
- uses: actions/checkout@v2
- name: cml_run
Expand Down
5 changes: 3 additions & 2 deletions content/docs/start/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ We built CML with these principles in mind:
plots in each Git Pull Request. Rigorous engineering practices help your team
make informed, data-driven decisions.
- **No additional services.** Build your own ML platform using just GitHub or
GitLab and your favorite cloud services: AWS, Azure, GCP. No databases,
services or complex setup needed.
GitLab and your
[favorite cloud services](/doc/self-hosted-runners#runner-credentials): AWS,
Azure, GCP, or Kubernetes. No databases, services or complex setup needed.

_Need help? Just want to chat about continuous integration for ML?
[Visit our Discord channel!](https://discord.gg/bzA6uY7)_
Expand Down
13 changes: 5 additions & 8 deletions content/docs/usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,8 @@ on: [push]
jobs:
run:
runs-on: ubuntu-latest
# optionally use a convenient Ubuntu LTS + CUDA + DVC + CML image
# container: docker://iterativeai/cml:0-dvc2-base1-gpu
# container: docker://ghcr.io/iterative/cml:0-dvc2-base1-gpu
# optionally use a convenient Ubuntu LTS + DVC + CML container
# container: docker://ghcr.io/iterative/cml:0-dvc2-base1
steps:
- uses: actions/checkout@v2
# may need to setup NodeJS & Python3 on e.g. self-hosted
Expand All @@ -58,12 +57,10 @@ jobs:

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
example, uncommenting the field
`container: docker://iterativeai/cml:0-dvc2-base1-gpu` (or
`container: docker://ghcr.io/iterative/cml:0-dvc2-base1-gpu`) will make the
example, uncommenting the
`container: docker://ghcr.io/iterative/cml:0-dvc2-base1` field will make the
runner pull the CML Docker image. The image already has NodeJS, Python 3, DVC
and CML set up on an Ubuntu LTS base with CUDA libraries and
[Terraform](https://www.terraform.io) installed for convenience.
and CML set up on an Ubuntu LTS base for convenience.

## CML Functions

Expand Down
Loading