Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix modelsave #52

Merged
merged 4 commits into from
Apr 19, 2018
Merged

Fix modelsave #52

merged 4 commits into from
Apr 19, 2018

Conversation

YujiOshima
Copy link
Contributor

Fix bug that the last trial of the study won't be saved when a study has completed.
I changed to save models of all completed trials before breaking trialIteration.

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>
Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>
@gaocegege
Copy link
Member

/assign @mitake

For the changes in dlk.

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>
err := lt.checkPodStatus(podState)
if err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
lt.pollJobs()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the motivation of swapping pollJobs() and checkPodStatus()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If pollJobs is called before checkPodStatus, when logs of pod updated and the pod completed after pollJobs was called, the last logs won't be collected by dlk.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understood, thanks!

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>
@mitake
Copy link
Contributor

mitake commented Apr 19, 2018

@gaocegege the change of dlk side looks good to me

@gaocegege
Copy link
Member

/lgtm

@mitake Thanks :-)

@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gaocegege

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 56e143a into kubeflow:master Apr 19, 2018
@YujiOshima YujiOshima deleted the fix-modelsave branch April 24, 2018 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants