Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor Examples folder structure #1691

Merged
merged 19 commits into from
Oct 7, 2021
Merged

Refactor Examples folder structure #1691

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
1403dc1
Refactor Katib Examples
andreyvelich Sep 30, 2021
80e9305
Fix links
andreyvelich Oct 2, 2021
1515597
Use Kind image
andreyvelich Oct 4, 2021
707b510
Update examples/v1beta1/kind-cluster/README.md
andreyvelich Oct 4, 2021
9a31e5e
Update examples/v1beta1/kind-cluster/README.md
andreyvelich Oct 4, 2021
a8b36c2
Increase timeout
andreyvelich Oct 4, 2021
c784b37
Update docs/images-location.md
andreyvelich Oct 4, 2021
a2c5a61
Update examples/v1beta1/README.md
andreyvelich Oct 4, 2021
5ffb727
Remove json
andreyvelich Oct 4, 2021
31130f7
Add example links to training containers
andreyvelich Oct 5, 2021
6791fc9
Fix link
andreyvelich Oct 5, 2021
3739ff0
Update links to training-operator
andreyvelich Oct 5, 2021
1bd2ffb
Rename Trial settings to template
andreyvelich Oct 6, 2021
c822040
Rename Trial training containers to Trial images
andreyvelich Oct 6, 2021
b575b62
Move NAS examples to Trial images
andreyvelich Oct 6, 2021
476c9cd
Add NAS links to README
andreyvelich Oct 6, 2021
ce8da88
Change TARGET DIR
andreyvelich Oct 6, 2021
8881113
Update examples/v1beta1/trial-images/mxnet-mnist/Dockerfile
andreyvelich Oct 7, 2021
2dd70ff
Update examples/v1beta1/trial-images/pytorch-mnist/Dockerfile
andreyvelich Oct 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test-charmed-katib.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
- name: Test Katib
run: |
set -eux
kubectl apply -f examples/v1beta1/random-example.yaml
kubectl apply -f examples/v1beta1/hp-tuning/random.yaml

- name: Get pod statuses
run: kubectl get all -A
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ katib-mysql-7894994f88-5d4s5 1/1 Running 0 36s
katib-ui-5767cfccdc-pwg2x 1/1 Running 0 36s
```

For the Katib Experiments check the [complete examples list](examples).
For the Katib Experiments check the [complete examples list](./examples/v1beta1).

# Documentation

Expand Down
362 changes: 362 additions & 0 deletions docs/images-location.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,362 @@
# Katib Images Location

Here you can find the location for images that are used in Katib.

## Katib Components Images

The following table shows images for the
[Katib components](https://www.kubeflow.org/docs/components/katib/hyperparameter/#katib-components).

<table>
<tbody>
<tr align="center">
<td>
<b>Image Name</b>
</td>
<td>
<b>Description</b>
</td>
<td>
<b>Location</b>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/katib-controller</code>
</td>
<td>
Katib Controller
</td>
<td>
<a href="https://github.com/kubeflow/katib/tree/master/cmd/katib-controller/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/katib-ui</code>
</td>
<td>
Katib User Interface
</td>
<td>
<a href="https://github.com/kubeflow/katib/tree/master/cmd/ui/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/katib-db-manager</code>
</td>
<td>
Katib DB Manager
</td>
<td>
<a href="https://github.com/kubeflow/katib/tree/master/cmd/db-manager/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/mysql</code>
</td>
<td>
Katib MySQL DB
</td>
<td>
<a href="https://github.com/docker-library/mysql/blob/c506174eab8ae160f56483e8d72410f8f1e1470f/8.0/Dockerfile.debian">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/katib-cert-generator</code>
</td>
<td>
Katib Cert Generator
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/cert-generator/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
</tbody>
</table>

## Katib Metrics Collectors Images

The following table shows images for the
[Katib Metrics Collectors](https://www.kubeflow.org/docs/components/katib/experiment/#metrics-collector).

<table>
<tbody>
<tr align="center">
<td>
<b>Image Name</b>
</td>
<td>
<b>Description</b>
</td>
<td>
<b>Location</b>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/file-metrics-collector</code>
</td>
<td>
File Metrics Collector
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/metricscollector/v1beta1/file-metricscollector/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/tfevent-metrics-collector</code>
</td>
<td>
Tensorflow Event Metrics Collector
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/metricscollector/v1beta1/tfevent-metricscollector/Dockerfile">Dockerfile</a>
</td>
</tr>
</tbody>
</table>

## Katib Suggestions and Early Stopping Images

The following table shows images for the
[Katib Suggestions](https://www.kubeflow.org/docs/components/katib/experiment/#search-algorithms-in-detail)
and the [Katib Early Stopping algorithms](https://www.kubeflow.org/docs/components/katib/early-stopping/).

<table>
<tbody>
<tr align="center">
<td>
<b>Image Name</b>
</td>
<td>
<b>Description</b>
</td>
<td>
<b>Location</b>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-hyperopt</code>
</td>
<td>
<a href="https://github.com/hyperopt/hyperopt">Hyperopt</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/hyperopt/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-chocolate</code>
</td>
<td>
<a href="https://github.com/AIworx-Labs/chocolate">Chocolate</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/chocolate/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-skopt</code>
</td>
<td>
<a href="https://github.com/scikit-optimize/scikit-optimize">Skopt</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/skopt/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-optuna</code>
</td>
<td>
<a href="https://github.com/optuna/optuna">Optuna</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/optuna/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-goptuna</code>
</td>
<td>
<a href="https://github.com/c-bata/goptuna">Goptuna</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/goptuna/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-hyperband</code>
</td>
<td>
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#hyperband">Hyperband</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/hyperband/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-enas</code>
</td>
<td>
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#enas">ENAS</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/nas/enas/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/suggestion-darts</code>
</td>
<td>
<a href="https://www.kubeflow.org/docs/components/katib/experiment/#differentiable-architecture-search-darts">DARTS</a> Suggestion
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/suggestion/nas/darts/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/earlystopping-medianstop</code>
</td>
<td>
<a href="https://www.kubeflow.org/docs/components/katib/early-stopping/#median-stopping-rule">Median Stopping Rule</a>
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/cmd/earlystopping/medianstop/v1beta1/Dockerfile">Dockerfile</a>
</td>
</tr>
</tbody>
</table>

## Training Containers Images

The following table shows images for training containers which are used in the
[Katib Trials](https://www.kubeflow.org/docs/components/katib/experiment/#packaging-your-training-code-in-a-container-image).

<table>
<tbody>
<tr align="center">
<td>
<b>Image Name</b>
</td>
<td>
<b>Description</b>
</td>
<td>
<b>Location</b>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/mxnet-mnist</code>
</td>
<td>
MXNet MNIST example with collecting metrics time
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/mxnet-mnist/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/pytorch-mnist</code>
</td>
<td>
PyTorch MNIST example with printing metrics to the file or StdOut
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/pytorch-mnist/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>gcr.io/kubeflow-ci/tf-mnist-with-summaries</code>
</td>
<td>
Tensorflow MNIST example with saving metrics in the summaries
</td>
<td>
<a href="https://github.com/kubeflow/training-operator/blob/2712f5667ec78f17d22288630f8719f0c08990ba/examples/tensorflow/mnist_with_summaries/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/xgboost-lightgbm</code>
</td>
<td>
Distributed LightGBM example for XGBoostJob
</td>
<td>
<a href="https://github.com/kubeflow/xgboost-operator/blob/9c8c97d0125a8156f12b8ef5b93f99e709fb57ea/config/samples/lightgbm-dist/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflow/mpi-horovod-mnist</code>
</td>
<td>
Distributed Horovod example for MPIJob
</td>
<td>
<a href="https://github.com/kubeflow/mpi-operator/blob/947d396a9caf70d3c94bf587d5e5da32b70f0f53/examples/horovod/Dockerfile.cpu">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/inaccel/jupyter:lab</code>
</td>
<td>
FPGA XGBoost with parameter tuning
</td>
<td>
<a href="https://github.com/inaccel/jupyter/blob/master/lab/Dockerfile">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/enas-cnn-cifar10-gpu</code>
</td>
<td>
Keras CIFAR-10 CNN example for ENAS with GPU support
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/enas-cnn-cifar10/Dockerfile.gpu">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/enas-cnn-cifar10-cpu</code>
</td>
<td>
Keras CIFAR-10 CNN example for ENAS with CPU support
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/enas-cnn-cifar10/Dockerfile.cpu">Dockerfile</a>
</td>
</tr>
<tr align="center">
<td>
<code>docker.io/kubeflowkatib/darts-cnn-cifar10</code>
</td>
<td>
PyTorch CIFAR-10 CNN example for DARTS
</td>
<td>
<a href="https://github.com/kubeflow/katib/blob/master/examples/v1beta1/trial-images/darts-cnn-cifar10/Dockerfile">Dockerfile</a>
</td>
</tr>
</table>
Loading