helm pull oci not using --insecure-skip-tls-verify on self-signed registries

[koshrf]

Checklist:

• [ X] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [ X] I've included steps to reproduce the bug.
• [ X ] I've pasted the output of argocd version.

Describe the bug

To Reproduce

Add the private helm repo registry:

argocd repo add  harbor.lab.example --insecure-skip-server-verification --type helm --name pruebaoci --enable-oci --username USER --password PASSWORD

argocd repo list -o yaml                                                                                                                    20 ↵
- connectionState:
    attemptedAt: "2023-04-28T17:54:53Z"
    message: ""
    status: Successful
  enableOCI: true
  insecure: true
  name: pruebaoci
  repo: harbor.lab.example
  type: helm
  username: prueba

Create the app:

argocd app create pruebasoci --repo harbor.lab.example --insecure --helm-chart pruebasoci/pruebaoci --revision 0.1.0 --dest-namespace pruebas --dest-server https://kubernetes.default.svc

FATA[0000] rpc error: code = InvalidArgument desc = application spec for pruebasoci is invalid: InvalidSpecError: Unable to generate manifests in : rpc error: code = Unknown desc = `helm pull oci://harbor.lab.example/pruebasoci/pruebaoci --version 0.1.0 --destination /tmp/e1bb0fd5-9224-4187-bad9-d49171e1b244` failed exit status 1: Error: failed to do request: Head "https://harbor.lab.example/v2/pruebasoci/pruebaoci/manifests/0.1.0": x509: certificate signed by unknown authority 

It is the same using the UI:

Unable to create application: application spec for PruebaOCI is invalid: InvalidSpecError: Unable to generate manifests in : rpc error: code = Unknown desc = `helm pull oci://harbor.lab.example/pruebasoci/pruebaoci --version 0.1.0 --destination /tmp/577aea91-0681-407e-997d-47e086eaf5ef` failed exit status 1: Error: failed to do request: Head "https://harbor.lab.example/v2/pruebasoci/pruebaoci/manifests/0.1.0": x509: certificate signed by unknown authority

Expected behavior

Argocd to do a helm pull of a self-signed harbor helm oci registry.

Screenshots

Version

argocd: v2.6.7+5bcd846
  BuildDate: 2023-03-23T15:24:49Z
  GitCommit: 5bcd846fa16e4b19d8f477de7da50ec0aef320e5
  GitTreeState: clean
  GoVersion: go1.18.10
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.6.7+5bcd846
  BuildDate: 2023-03-23T14:57:27Z
  GitCommit: 5bcd846fa16e4b19d8f477de7da50ec0aef320e5
  GitTreeState: clean
  GoVersion: go1.18.10
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v4.5.7 2022-08-02T16:35:54Z
  Helm Version: v3.10.3+g835b733
  Kubectl Version: v0.24.2
  Jsonnet Version: v0.19.1

Logs

On the log of the pod:

time="2023-04-28T18:11:56Z" level=info msg="helm registry login harbor.lab.example --username ****** --password ****** --insecure" dir= execID=83797
time="2023-04-28T18:11:56Z" level=info msg=Trace args="[helm registry login harbor.lab.example --username ****** --password ****** --insecure]" dir= operation_name="exec helm" time_ms=111.135265
time="2023-04-28T18:11:56Z" level=info msg="helm pull oci://harbor.lab.example/pruebasoci/pruebaoci --version 0.1.0 --destination /tmp/e1bb0fd5-9224-4187-bad9-d49171e1b244" dir= execID=65512
time="2023-04-28T18:11:56Z" level=error msg="`helm pull oci://harbor.lab.example/pruebasoci/pruebaoci --version 0.1.0 --destination /tmp/e1bb0fd5-9224-4187-bad9-d49171e1b244` failed exit status 1: Error: failed to do request: Head \"https://harbor.lab.example/v2/pruebasoci/pruebaoci/manifests/0.1.0\": x509: certificate signed by unknown authority" execID=65512

It is able to do 'login' and use the --insecure flag, but when trying to do a pull it doesn't include the --insecure-skip-tls-verify flag required by helm.

[koshrf]

A similar bug is here: #12371

Still no answers from the dev :( can't use private helm registries with argocd

I also tried to add the TLS certificate but argocd just ignore it.

[guguducken]

@koshrf do you solve it? I have met same error :(

[RobertDiebels]

@koshrf @guguducken I ran into the same issue. Looks like the insecure flag was added in v2.8.1[ https://github.com/codefresh-io/argo-cd/commit/d14f496e87f30c40e14a5810e72cf39ca54fca8c ]

And the accompanying CAPath flag was added in 2.8.3 [ https://github.com/argoproj/argo-cd/commit/8c0456be18f5245c720a3159c3dc96855dfcef17 ]

Documentation has not been updated so I had way more trouble to find out what to do then should have been the case.

[augeivv]

We have the problem as well and cannot continue without the feature.
We use the current version 2.8.4.

Is there a solution for this? @RobertDiebels

[RobertDiebels]

We have the problem as well and cannot continue without the feature. We use the current version 2.8.4.

Is there a solution for this? @RobertDiebels

I was unable to verify whether the fixes I mentioned resolved the issue. This is due to our team being bound to an update-cycle beyond our control. As far as I know updating to 2.8.4 should fix the issue. We will probably revisit the issue once ArgoCD is updated in our cycle.

[bigg01]

We are on 2.7.14+a40c95a.dirtyArgo - same problem

[djajcevic]

Hi!

Any updates on this?

With the latest version as of today we are facing the same problem.

[avt548499]

We've got the same issue, any updates?