Update workflow to work on Istio enabled namespace

[omerfsen]

Update workflow to work on Istio enabled namespaces because workflow-controller created pods should have no sidecar otherwise it waits forever on first step

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

[seldondev]

Hi @omerfsen. Thanks for your PR.

I'm waiting for a SeldonIO member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

[omerfsen]

Thank you @groszewn for helping me on sorting out this issue!

[ryandawsonuk]

I remember this annotation cropping up with kubeflow pipelines, I presume for the same reason. https://github.com/kubeflow/pipelines/blob/fe21cd2dae4fcae6d5a28dcfd8346370a644537f/backend/src/apiserver/resource/resource_manager.go#L344 There was no real downside to introducing it as far as I remember. I think you possibly lose MTLS/security policies but you prob wouldn’t want that with a pipeline anyway.
/lgtm
/approve

[ryandawsonuk]

/ok-to-test

[seldondev]

Tue Nov 17 08:24:20 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2636/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2636 --build=2

[seldondev]

Tue Nov 17 08:24:29 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2636/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2636 --build=1

[ryandawsonuk]

/approve

[seldondev]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ryandawsonuk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ryandawsonuk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[axsaucedo]

This is great! This also fixes the issue you reported when running the batch notebook on istio enabled namespaces as well as other potential corner cases. Strange it hasn't come up more often but I guess many people running istio may not be running it with proxy injection enabled.