Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support traffic settings for shadow deployment with istio #3780

Merged
merged 1 commit into from
Mar 17, 2022

Conversation

ChowXu
Copy link
Contributor

@ChowXu ChowXu commented Dec 2, 2021

What this PR does / why we need it: support traffic settings for shadow deployment with istio to meet more situations

Special notes for your reviewer:

Which issue(s) this PR fixes: Fixes #3772

Does this PR introduce a user-facing change?: NONE

@seldondev
Copy link
Collaborator

Hi @ChowXu. Thanks for your PR.

I'm waiting for a SeldonIO or todo member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

@ukclivecox
Copy link
Contributor

ukclivecox commented Dec 2, 2021

Thanks @ChowXu
I think we would also need to modify the validating webhook check for traffic to ignore the shadow value in checking traffic sums to 100. We might also want to check that the shadow traffic percentrage is >=0 <= 100.

func checkTraffic(spec *SeldonDeploymentSpec, fldPath *field.Path, allErrs field.ErrorList) field.ErrorList {
var trafficSum int32 = 0
var shadows int = 0
for i := 0; i < len(spec.Predictors); i++ {
p := spec.Predictors[i]
trafficSum = trafficSum + p.Traffic
if p.Shadow == true {
shadows += 1
if shadows > 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[i].Name, "Multiple shadows are not allowed"))
}
}
}
if trafficSum != 100 && (len(spec.Predictors)-shadows) > 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[0].Name, "Traffic must sum to 100 for multiple predictors"))
}
if trafficSum > 0 && trafficSum < 100 && len(spec.Predictors) == 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[0].Name, "Traffic must sum be 100 for a single predictor when set"))
}
return allErrs
}

@ChowXu ChowXu force-pushed the xuzhou/istio_traffic_settings branch 6 times, most recently from 0d5206c to 8424ef3 Compare December 2, 2021 13:12
@ChowXu
Copy link
Contributor Author

ChowXu commented Dec 2, 2021

Thanks @ChowXu I think we would also need to modify the validating webhook check for traffic to ignore the shadow value in checking traffic sums to 100. We might also want to check that the shadow traffic percentrage is >=0 <= 100.

func checkTraffic(spec *SeldonDeploymentSpec, fldPath *field.Path, allErrs field.ErrorList) field.ErrorList {
var trafficSum int32 = 0
var shadows int = 0
for i := 0; i < len(spec.Predictors); i++ {
p := spec.Predictors[i]
trafficSum = trafficSum + p.Traffic
if p.Shadow == true {
shadows += 1
if shadows > 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[i].Name, "Multiple shadows are not allowed"))
}
}
}
if trafficSum != 100 && (len(spec.Predictors)-shadows) > 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[0].Name, "Traffic must sum to 100 for multiple predictors"))
}
if trafficSum > 0 && trafficSum < 100 && len(spec.Predictors) == 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[0].Name, "Traffic must sum be 100 for a single predictor when set"))
}
return allErrs
}

Thanks @cliveseldon , the checkTraffic's logical has been updated

}
if trafficSum > 0 && trafficSum < 100 && len(spec.Predictors) == 1 {
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[0].Name, "Traffic must sum be 100 for a single predictor when set"))
if shadows == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to do the checks even if shadows are present. We just need to ignore the shadow percentage in the calculation.

@axsaucedo
Copy link
Contributor

Looks good - following up on some of the requests, it would be great if you could also add some docs, and potentially extend some of the istio docs to show an example (i.e. the istio notebook)

@ChowXu ChowXu force-pushed the xuzhou/istio_traffic_settings branch from 8424ef3 to 306f927 Compare December 3, 2021 06:30
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@seldondev seldondev added size/L and removed size/S labels Dec 5, 2021
@ChowXu ChowXu force-pushed the xuzhou/istio_traffic_settings branch from c8f71ee to 0f7ede8 Compare December 5, 2021 15:00
@ChowXu
Copy link
Contributor Author

ChowXu commented Dec 5, 2021

thanks @axsaucedo , i have added the shadow deploy doc upder examples.istio , and also with a tip about traffic settings usage.

@ukclivecox
Copy link
Contributor

@ChowXu There are some errors compiling the operator.

@ChowXu ChowXu force-pushed the xuzhou/istio_traffic_settings branch from 0f7ede8 to e835564 Compare December 5, 2021 16:07
@ChowXu
Copy link
Contributor Author

ChowXu commented Dec 5, 2021

thanks @cliveseldon , it fixed now

@ChowXu ChowXu requested a review from ukclivecox December 7, 2021 10:31
allErrs = append(allErrs, field.Invalid(fldPath, spec.Predictors[i].Name, "shadow traffic is illegal, the traffic number should be between [0, 100]"))
}
} else {
trafficSum = trafficSum + p.Traffic
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a go test for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, i have add a UT for shadow traffic validation

@ChowXu ChowXu force-pushed the xuzhou/istio_traffic_settings branch from e835564 to 02cc652 Compare December 9, 2021 12:57
@seldondev seldondev added size/XL and removed size/L labels Dec 9, 2021
@ChowXu ChowXu requested a review from ukclivecox December 14, 2021 01:22
@axsaucedo axsaucedo self-requested a review February 18, 2022 10:06
@ukclivecox
Copy link
Contributor

/test integration

@ukclivecox
Copy link
Contributor

/test notebook

@seldondev
Copy link
Collaborator

@ChowXu: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test integration

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

@ChowXu
Copy link
Contributor Author

ChowXu commented Feb 25, 2022

/retest

@seldondev
Copy link
Collaborator

@ChowXu: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

@ChowXu
Copy link
Contributor Author

ChowXu commented Feb 25, 2022

Hi @axsaucedo the integration test meet some problems, but i have not authorization to visit pipelinerun page to see detail about what happended and i cannot rerun the test , could u give me a help?

@ukclivecox
Copy link
Contributor

/test integration

@ukclivecox
Copy link
Contributor

@ChowXu will run again as failed early so need to see if real issue

@axsaucedo
Copy link
Contributor

/test integration

1 similar comment
@axsaucedo
Copy link
Contributor

/test integration

@axsaucedo
Copy link
Contributor

/approve

@seldondev
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: axsaucedo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@axsaucedo axsaucedo merged commit 50145a4 into SeldonIO:master Mar 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Traffic Settings to SeldonDeployment For Shadow Deployment with istio
4 participants