Update Argo to v2.6

[nakfour]

What steps did you take:

We are trying to run KF pipelines on Openshift, this is not a bug but an enhancement. We are asking for an Argo version upgrade mainly for three reasons:

1. We ran into this issue that was fixed in Argo 2.4 Unable to write output artifacts when service account has limited permissions argoproj/argo-workflows#1445
2. Argo 2.5 introduced "workflow.failures" that is used to track workflow failures properly: https://github.com/argoproj/argo/blob/master/docs/variables.md#exit-handler
   feat: Add failed node info to exit handler argoproj/argo-workflows#2166
3. Argo 2.3 is pretty out dated since they are on Argo 2.6.x right now.

[Bobgy]

/assign @Ark-kun
/cc @IronPan @chensun @jingzhang36 @rmgogogo
@Ark-kun I remember you had a PR trying to upgrade argo, where was it blocked?

For me, the fix for argoproj/argo-workflows#1422 introduced in argo 2.4.x was a step forward for enabling istio sidecar in workflows.

[Bobgy]

Also
/cc @neuromage if you have any concerns for dsl related issues

[Ark-kun]

@Ark-kun I remember you had a PR trying to upgrade argo, where was it blocked?

The upgrade worked, but there wasn't enough consensus that there is a need to upgrade #2419 .

I think it might be good to upgrade. There are several important fixes in later versions.

[Bobgy]

Thanks for sharing the context! Looking at the PR I assume there won't be a lot of changes needed.
Currently put this in post 1.0 plan, but will relook at it if we have extra bandwidth.

[Jeffwan]

@Bobgy @Ark-kun

Vote for Argo upgrade. There's critical improvement for AWS users in https://github.com/argoproj/argo/releases/tag/v2.5.0. We need
#1899 feat(artifacts): Add support for IAM roles for service accounts in artifact storage. Fixes #1774 this feature to inject role, otherwise user always need to use AWS credentials which are not secure.

I am not sure if current 2.3 -> 2.6 is a big jump, seems minimum version @nakfour need is 2.5.x as well. I think for KFP 1.0, we at least want to support 2.5.x.

@nakfour Can you confirm which exact version brings "workflow.failures""?

[Bobgy]

@hongye-sun it seems this is one more reason for this upgrade: #3590