Adding initial v1alpha2 API controller #457

johnugeorge · 2019-04-15T09:19:43Z

This is an initial version of level based implementation of controller for early review of code flow. I am waiting for #435 and #456 to get merged first and make necessary changes after that.

Invalid conditions are not validated yet.
Metric collector code is not added yet

Fixes: #441

/cc @richardsliu @gaocegege @YujiOshima

This change is

pkg/api/operators/apis/experiment/v1alpha2/register.go

pkg/api/operators/apis/trial/v1alpha2/register.go

pkg/controller/v1alpha2/experiment/experiment_controller.go

pkg/controller/v1alpha2/experiment/util/status_util.go

pkg/controller/v1alpha2/trial/util/status_util.go

pkg/controller/v1alpha2/experiment/experiment_controller.go

pkg/controller/v1alpha2/experiment/util/api_util.go

pkg/controller/v1alpha2/trial/util/api_util.go

pkg/controller/v1alpha2/experiment/util/status_util.go

richardsliu · 2019-04-16T19:56:02Z

pkg/controller/v1alpha2/trial/util/status_util.go

+				log.Info("Error in converting unstructured to status: %v ", err)
+				return err
+			}
+			if jobStatus.Active == 0 && jobStatus.Succeeded > 0 {


I think batch job supports JobCondition now: https://github.com/kubernetes/api/blob/master/batch/v1/types.go#L168

pkg/controller/v1alpha2/trial/util/status_util.go

pkg/controller/v1alpha2/experiment/experiment_controller.go

pkg/api/operators/apis/experiment/v1alpha2/experiment_types.go

pkg/api/operators/apis/experiment/v1alpha2/util.go

hougangliu · 2019-04-18T06:25:36Z

pkg/api/operators/apis/experiment/v1alpha2/util.go

+func (exp *Experiment) MarkExperimentStatusSucceeded(reason, message string) {
+	currentCond := getCondition(exp, ExperimentRunning)
+	if currentCond != nil {
+		exp.setCondition(ExperimentRunning, v1.ConditionFalse, currentCond.Reason, currentCond.Message)


Why mark ExperimentRunning as v1.ConditionFalse?
and why not mark ExperimentCreated as v1.ConditionFalse when MarkExperimentStatusRunning?

I tried to follow the same conventions from the job operators. But, it is true that we have to rethink on making more meaningful conditions. eg: we may not need Successful condition and Running condition. We could make Running as false stating reason as Successful. This is followed in k8s default objects

pkg/controller/v1alpha2/experiment/util/status_util.go

pkg/controller/v1alpha2/experiment/experiment_controller.go

cmd/katib-controller/v1alpha2/main.go

pkg/controller/v1alpha2/experiment/experiment_controller.go

johnugeorge · 2019-04-22T18:21:59Z

/retest

andreyvelich · 2019-04-23T00:31:46Z

pkg/controller/v1alpha2/experiment/experiment_controller.go

+func (r *ReconcileExperiment) createTrials(instance *experimentsv1alpha2.Experiment, addCount int) error {
+
+	logger := log.WithValues("Experiment", types.NamespacedName{Name: instance.GetName(), Namespace: instance.GetNamespace()})
+	trials, err := util.GetSuggestions(instance, addCount)


@johnugeorge
It can run more than one Suggestion at once, since we name it GetSuggestions?

Can you please explain more?

I mean you have named this function GetSuggestions. Is it make sense? I thought we should name it GetSuggestion.

pkg/controller/v1alpha2/experiment/util/status_util.go

pkg/controller/v1alpha2/trial/util/status_util.go

hougangliu · 2019-04-23T00:23:23Z

pkg/controller/v1alpha2/experiment/experiment_controller.go

+	completedCount := instance.Status.TrialsSucceeded + instance.Status.TrialsFailed + instance.Status.TrialsKilled
+
+	if activeCount > parallelCount {
+		deleteCount := activeCount - parallelCount


I think line 215 and 223 can change to if activeCount >= parallelCount {} else {}

hougangliu · 2019-04-23T00:28:39Z

pkg/controller/v1alpha2/experiment/experiment_controller.go

+		if deleteCount > 0 {
+			//delete 'deleteCount' number of trails. Sort them?
+			logger.Info("DeleteTrials", "deleteCount", deleteCount)
+			err = r.deleteTrials(instance, deleteCount)


r.deleteTrials will make Status.TrialsKilled? will it make completedCount value dirty in line 213?

I feel that TrialsKilled has a different meaning. We should just delete the trial here.

pkg/controller/v1alpha2/experiment/experiment_controller.go

pkg/controller/v1alpha2/trial/trial_controller.go

hougangliu · 2019-04-23T01:14:25Z

@richardsliu @YujiOshima @johnugeorge @gaocegege how about merging this PR first? Then we can code further based on it.

gaocegege · 2019-04-23T01:28:38Z

SGTM.

/lgtm

andreyvelich · 2019-04-23T01:34:29Z

@hougangliu @gaocegege @richardsliu @YujiOshima @johnugeorge @gaocegege
Should we split the work on the Controller in to small issues, after we merge this PR?

YujiOshima · 2019-04-23T02:51:51Z

pkg/controller/v1alpha2/trial/trial_controller.go

+			instance.Status.StartTime = &now
+		}
+		msg := "Trial is created"
+		instance.MarkTrialStatusCreated(util.TrialCreatedReason, msg)


MarkTrialStatusCreated is collect? It looks you need to mark running for going next step.

Trial is marked running when jobs are in running state

YujiOshima · 2019-04-23T02:55:12Z

pkg/controller/v1alpha2/experiment/experiment_controller.go

+			instance.Status.StartTime = &now
+		}
+		msg := "Experiment is created"
+		instance.MarkExperimentStatusCreated(util.ExperimentCreatedReason, msg)


MarkTrialStatusCreated is collect? It looks you need to mark running for going next step.

When the Experiment is created, it is marked Created. It is marked running when trials are created.

YujiOshima · 2019-04-23T03:06:10Z

@hougangliu @andreyvelich Thanks, I agree with we proceed step by step since this change will be big and complicated.
@johnugeorge Does it OK?

johnugeorge · 2019-04-23T12:09:52Z

I have verified creation of experiment and trial objects with status updates. We can merge this PR if current review is completed.

hougangliu · 2019-04-23T23:20:47Z

/lgtm
/approve

k8s-ci-robot · 2019-04-23T23:20:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hougangliu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hougangliu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Adding initial v1alpha2 API controller (kubeflow#457) * Adding initial v1alpha2 controller * Adding logs * Adding comments * Adding template functions for experiment * Adding error checks * chore: Skip test when code is not changed (kubeflow#467) Signed-off-by: Ce Gao <gaoce@caicloud.io> * Add serviceAccountName in UI deployment (kubeflow#469) * feat: Introduce suggestion Signed-off-by: Ce Gao <gaoce@caicloud.io> * owners: Add xinyi yu Signed-off-by: Ce Gao <gaoce@caicloud.io>

k8s-ci-robot requested review from gaocegege, richardsliu and YujiOshima April 15, 2019 09:19

k8s-ci-robot added do-not-merge/work-in-progress size/XXL labels Apr 15, 2019

gaocegege reviewed Apr 15, 2019

View reviewed changes

andreyvelich reviewed Apr 15, 2019

View reviewed changes

pkg/controller/v1alpha2/experiment/experiment_controller.go Outdated Show resolved Hide resolved

richardsliu mentioned this pull request Apr 16, 2019

StudyJob v1alpha2 API version #370

Closed

15 tasks

richardsliu reviewed Apr 16, 2019

View reviewed changes

hougangliu reviewed Apr 18, 2019

View reviewed changes

pkg/controller/v1alpha2/experiment/util/status_util.go Show resolved Hide resolved

pkg/controller/v1alpha2/experiment/experiment_controller.go Show resolved Hide resolved

johnugeorge added 3 commits April 19, 2019 14:48

Adding initial v1alpha2 controller

7a8c0ee

Adding logs

cde0287

Adding comments

8c48f16

johnugeorge force-pushed the controller-v1alpha2 branch from 04b1cc4 to 8c48f16 Compare April 21, 2019 11:02

hougangliu reviewed Apr 21, 2019

View reviewed changes

cmd/katib-controller/v1alpha2/main.go Outdated Show resolved Hide resolved

pkg/controller/v1alpha2/experiment/experiment_controller.go Outdated Show resolved Hide resolved

johnugeorge mentioned this pull request Apr 22, 2019

[v1alpha2] Controller implementation #441

Closed

Adding template functions for experiment

169b03c

johnugeorge changed the title ~~WIP: Adding initial v1alpha2 API controller~~ Adding initial v1alpha2 API controller Apr 22, 2019

k8s-ci-robot removed the do-not-merge/work-in-progress label Apr 22, 2019

andreyvelich reviewed Apr 23, 2019

View reviewed changes

pkg/controller/v1alpha2/experiment/util/status_util.go Show resolved Hide resolved

pkg/controller/v1alpha2/trial/util/status_util.go Show resolved Hide resolved

hougangliu reviewed Apr 23, 2019

View reviewed changes

pkg/controller/v1alpha2/experiment/experiment_controller.go Outdated Show resolved Hide resolved

pkg/controller/v1alpha2/experiment/experiment_controller.go Outdated Show resolved Hide resolved

pkg/controller/v1alpha2/trial/trial_controller.go Show resolved Hide resolved

k8s-ci-robot assigned gaocegege Apr 23, 2019

k8s-ci-robot added the lgtm label Apr 23, 2019

YujiOshima reviewed Apr 23, 2019

View reviewed changes

Adding error checks

e40b895

k8s-ci-robot removed the lgtm label Apr 23, 2019

k8s-ci-robot assigned hougangliu Apr 23, 2019

k8s-ci-robot added the lgtm label Apr 23, 2019

k8s-ci-robot added the approved label Apr 23, 2019

k8s-ci-robot merged commit 5a7a144 into kubeflow:master Apr 23, 2019

akgraner mentioned this pull request Jan 2, 2024

Kubeflow Steering Committee Elections - Testimonial Phase - Johnu George kubeflow/community#675

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding initial v1alpha2 API controller #457

Adding initial v1alpha2 API controller #457

johnugeorge commented Apr 15, 2019 •

edited

Loading

richardsliu Apr 16, 2019

hougangliu Apr 18, 2019

johnugeorge Apr 22, 2019

johnugeorge commented Apr 22, 2019

andreyvelich Apr 23, 2019

johnugeorge Apr 23, 2019

andreyvelich Apr 23, 2019

hougangliu Apr 23, 2019

hougangliu Apr 23, 2019 •

edited

Loading

johnugeorge Apr 23, 2019

hougangliu commented Apr 23, 2019

gaocegege commented Apr 23, 2019

andreyvelich commented Apr 23, 2019

YujiOshima Apr 23, 2019

johnugeorge Apr 23, 2019

YujiOshima Apr 23, 2019

johnugeorge Apr 23, 2019

YujiOshima commented Apr 23, 2019

johnugeorge commented Apr 23, 2019

hougangliu commented Apr 23, 2019

k8s-ci-robot commented Apr 23, 2019

Adding initial v1alpha2 API controller #457

Adding initial v1alpha2 API controller #457

Conversation

johnugeorge commented Apr 15, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnugeorge commented Apr 22, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hougangliu Apr 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hougangliu commented Apr 23, 2019

gaocegege commented Apr 23, 2019

andreyvelich commented Apr 23, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

YujiOshima commented Apr 23, 2019

johnugeorge commented Apr 23, 2019

hougangliu commented Apr 23, 2019

k8s-ci-robot commented Apr 23, 2019

johnugeorge commented Apr 15, 2019 •

edited

Loading

hougangliu Apr 23, 2019 •

edited

Loading