Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add 2020 roadmap #1121

Merged
merged 3 commits into from
Apr 7, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 18 additions & 30 deletions ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,31 @@
# Katib 2020 Roadmap

This document provides a high level view of where Katib will grow in 2020.
## New Features

The original Katib design document can be found [here](https://docs.google.com/document/d/1ZEKhou4z1utFTOgjzhSsnvysJFNEJmygllgDCBnYvm8/edit#heading=h.7fzqir88ovr).
### Hyperparameter Tuning

# Katib 1.0 Readiness
- Support Early Stopping [#692](https://github.com/kubeflow/katib/issues/692)

* Stabilize APIs for Experiments
* Reconsider the design of Trial Template [#906](https://github.com/kubeflow/katib/issues/906)
* Early Stopping [#692](https://github.com/kubeflow/katib/issues/692)
* Resuming Experiment [#1061](https://github.com/kubeflow/katib/issues/1061), [#1062](https://github.com/kubeflow/katib/issues/1062)
* Fully integrate Katib with existing E2E examples:
* Xgboost
* Mnist
* GitHub issue summarization
* Publish API documentation, best practices, tutorials
* [Issues list](https://github.com/kubeflow/katib/issues)
### Neural Architecture Search

# Enhance HP Tuning Experience
- Support Advanced NAS Algorithms like DARTs, ProxylessNAS [#461](https://github.com/kubeflow/katib/issues/461)

The objectives here are organized around the three stages defined in the CUJ:
### Other Features

## 1. Defining Model and Parameters
- Support Auto Model Compression [#460](https://github.com/kubeflow/katib/issues/460)
- Support Auto Feature Engineering [#475](https://github.com/kubeflow/katib/issues/475)

Integration with KF distributed training components
* TFJob
* PyTorch
* Allow Katib to support other operator types generically [#341](https://github.com/kubeflow/katib/issues/341)
## Enhancements

## 2. Configuring a Experiment
* Supporting additional suggestion algorithms [#15](https://github.com/kubeflow/katib/issues/15)
### Hyperparameter Tuning

## 3. Tracking Model Performance
* UI enhancements: allowing data scientists to visualize results easier
* Support for persistent model and metadata storage
* Ideally users should be able to export and reuse trained models from a common storage
- Delete Suggestion deployment after Experiment is finished [#1061](https://github.com/kubeflow/katib/issues/1061)
- Save Suggestion state after deployment is deleted [#1062](https://github.com/kubeflow/katib/issues/1062)
- Reconsider the design of Trial Template [#906](https://github.com/kubeflow/katib/issues/906)
- Add validation for algortihms (a.k.a suggestions) [#1126](https://github.com/kubeflow/katib/issues/1126)

# Test and Release Infrastructure
### Neural Architecture Search

* Improve e2e test coverage
* Improve test harness
* Enhance release process; adding automation (see https://bit.ly/2F7o4gM)
- Refactor structure for NAS algorithms [#1125](https://github.com/kubeflow/katib/issues/1125)
- Refactor the design for NAS model constructor[#1127](https://github.com/kubeflow/katib/issues/1127)
- Katib UI fixes and enhancements