Automate Container Build for Trial training containers

[andreyvelich]

/kind feature

We discussed in our WG meetings and in the summit on 2021-07-16 about Trial training container's source code automation.

Many users request automation for container build instead of manually building images. In that case, users will have uniform experience to tune HP from the Jupyter Notebooks.
We might extend the Katib SDK to support it.

Let's continue discussion about it under this issue.

Resources from GCP: using Buildpacks.

cc @gaocegege @johnugeorge
/priority p2

[johnugeorge]

Related Issue in Operators - kubeflow/common#66

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreyvelich]

/lifecycle frozen