Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to read additional environment variables within the argocd-repo-server #7189

Open
shanproofpoint opened this issue Sep 9, 2021 · 27 comments
Labels
enhancement New feature or request

Comments

@shanproofpoint
Copy link

shanproofpoint commented Sep 9, 2021

Summary

When specifying an Application yaml, we would like the argocd build envirionment to include all os level environment variables or can be restricted to any custom os environment variable prefixed with ARGOCD_

Motivation

we are setting up many clusters that poll a single git repo for their entire list of applications and their configuration. however, each cluster has an identity that is used in parameterizing some deployments and custom resources either through additional annotations, environment variables or other resources like disks. we store this as a configmap of values inside the argocd namespace which then can me mounted in the argocd-repo-server as environment variables.

Proposal

just read additional os level environment variables into the argocd build environment so we can do the following:

kind: Application
metadata:
  name: argocd-app-sops
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://isindir.github.io/sops-secrets-operator/
    chart: sops-secrets-operator
    targetRevision: 0.9.2
    helm:
      parameters:
        - name: "gcp.enabled"
          value: "true"
        - name: "gcp.existingSecretName"
          value: "sops-decrypter-access"
        - name: "podAnnotations.i-was-here/ok"
          value: $ARGOCD_MY_CUSTOM_ENV_VAR
        - name: "podAnnotations.i-was-here2/ok"
          value: $ARGOCD_APP_NAME

where ARGOCD_MY_CUSTOM_ENV_VAR would be read from a configmap into the argocd-repo-server container

@shanproofpoint shanproofpoint added the enhancement New feature or request label Sep 9, 2021
@Jmainguy
Copy link

We are currently using a plugin to get these vars, but that stops us from using the helm: block. So this would be a great feature to add.

@crenshaw-dev
Copy link
Member

I think this makes sense. Rather than prefixing with ARGOCD_ (there may be collisions with Argo CD env vars added in the future), maybe there could be an allow-list configurable via repo-server ConfigMap or env var?

@mstrYoda
Copy link

mstrYoda commented Oct 1, 2022

Hi, I would like to work on this issue. Can you assign it to me @crenshaw-dev ? Thanks.

@shanproofpoint
Copy link
Author

Hi, I would like to work on this issue. Can you assign it to me @crenshaw-dev ? Thanks.

thank you!

@jselleck-sans
Copy link

I'd like this too! In my case I want to override application placeholders with global (or config map) env vars. Our use case is gitops - we want to use the same manifests but pull from different branches depending on a cluster name variable (changing branch Revision). I also see us setting parameters in apps for helm charts - dependent on global params..

@jselleck-sans
Copy link

jselleck-sans commented Oct 21, 2022

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/argoproj/argocd-example-apps.git
    targetRevision: $MY_CUSTOM_ENV_VAR
    path: guestbook
  destination:
    server: https://kubernetes.default.svc
    namespace: guestbook
  • maybe add an annotation to the Application manifest to point to a CM for variables.

@SebastienTolron
Copy link

+1 Would be so good !

Trying to achieve the same goal. Having multiple cluster and 1 applicationSet. So , for ingress route, it would be pretty handfull.

@mstrYoda
Copy link

I finally find time to work on this. I will schedule it to develop with our Golang community 🙏

@shanproofpoint
Copy link
Author

shanproofpoint commented Jan 18, 2023 via email

@revaspeho
Copy link

Hi I am the head DevOps Engineer for the federal government. We need to be able to pass in our gitlab auth token into the url string for helm.valueFiles -https://$AUTH_TOKEN. This would be great!

@danhawkins
Copy link

👍 for this, I am trying to have a dynamic targetRevision also

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: letsencrypt
  namespace: argocd
spec:
  project: core-services
  source:
    path: services/letsencrypt/manifests
    repoURL: <repo>
    targetRevision: $TARGET_REVISION

@krzwiatrzyk
Copy link

@revaspeho theoretically you can achieve it with helm-secrets plugin - #7819
practically, environment substation doesn't work for me (but still trying to make it work)

@danhawkins
Copy link

@mstrYoda are you actively working on this?

@danhawkins
Copy link

I was able to get something to work here, I built my own docker image, removing this line from applicationset utils to allow access to the env function. Then using and applicationset with a go template

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: letsencrypt
  namespace: argocd
spec:
  goTemplate: true
  generators:
    - clusters: {}
  template:
    metadata:
      name: '{{.name}}-letsencrypt'
    spec:
      project: core-services
      source:
        path: services/letsencrypt/manifests
        repoURL: https://github.com/xxxxxxxxxxxx/argolab.git
        targetRevision: '{{ env "TARGET_REVISION" }}'
        directory:
          recurse: true
      destination:
        server: https://kubernetes.default.svc
        namespace: cert-manager
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
        automated:
          selfHeal: true

This works, but obviously there is some security concern. But I think if we are able to set an allowlist of vars somewhere like suggested earlier on, we can perhaps just filter out env values not on the list? Something like that. I mean I am sure there could be more elegant ways to solve this. But I am happy with this so far.

@danhawkins
Copy link

I have a PR that solves this, will submit it soon for review

@shanproofpoint
Copy link
Author

Thank you ! @danhawkins

@danhawkins
Copy link

@crenshaw-dev you can assign this to me, I made a PR, any feedback to get it merged would be appreciated.

@leoluz
Copy link
Collaborator

leoluz commented Apr 27, 2023

As mentioned in Contributor's meeting today, we aim to address this issue by the functionality implemented in this PR:
#12508

@pierluigilenoci
Copy link

@leoluz I looked at PR #12508 and needed help understanding how that can solve the problem proposed in this issue.
@shanproofpoint, what do you think about it?

@shanproofpoint
Copy link
Author

shanproofpoint commented May 22, 2023

@leoluz i also feel it does not directly address this need. We need something like a parameterized kustomization.yaml file each cluster that reads the same application yaml. Can you explain how multiple repos get merged?

@leoluz
Copy link
Collaborator

leoluz commented May 23, 2023

@shanproofpoint @pierluigilenoci With the functionality implemented in the multi-source PR a new concept of chained manifest generation will be possible. For example, this means that you will be able to rely on a helm chart to generate your basic manifests and then implement a small CMP plugin that does the last mile manifest manipulation required by your company.

@shanproofpoint
Copy link
Author

shanproofpoint commented May 23, 2023 via email

@krzwiatrzyk-lgd
Copy link

krzwiatrzyk-lgd commented Jul 27, 2023

@leoluz I think that chain manifest generation or CMP is an overkill for such need.

Showing another example: I am managing a fleet of EKS cluster and want to manage "infrastracture" apps via app-of-apps pattern. ArgoCD is deployed once on the cluster create and right away app-of-apps manifest pointing to infrastructure manfiests is deployed.

Infrastructure manifests contains an application manifest for cluster autoscaler which requires an unique clusterName to distinguish which autoscaling groups can be targetted for autoscaling.

Having environment substition in application manifest this will be very straightforward:

  1. Set environment on ArgoCD deployment
  2. Specify environment in application manifest.

Environment variables substitution is already there... but only for a small set of defined variables.

@AndreCbrera
Copy link

AndreCbrera commented Oct 5, 2023

may can you try envsubst '$ARGOCD_APP_NAME' < application.yaml | k apply -f - declaring before the variable export ARGOCD_APP_NAME="hellotherepoc" all this in argocd namespace

@maeghan-porter
Copy link

maeghan-porter commented Dec 19, 2023

I'm looking in to replacing fluxcd with argocd and I'm struggling with argocd not having the same idea of postBuild variable substitution that flux has. We have multiple multi-tenancy eks clusters for different environments (dev, beta, stable nonprod, stable prod), and we use variables such as the name of the cluster (needed for aws-specific helm charts), the environment, the stage (nonprod/prod), the vpc id the cluster is in, etc. via postBuild variables with flux. It would be nice if, in argocd, we could set some variables like this on the cluster objects and they could be rendered in the values for a helmchart or in kustomizations depending on which cluster it's being deployed to.

@AndreCbrera
Copy link

AndreCbrera commented Dec 19, 2023

@maeghan-porter may you can use some approach like envsubst command, I do this using helm values:

envsubst '$ARGOCD_APP_NAME' < values.yaml | helm install app-test .

@maeghan-porter
Copy link

maeghan-porter commented Dec 19, 2023

@maeghan-porter may you can use some approach like envsubst command, I do this using helm values:

envsubst '$ARGOCD_APP_NAME' < values.yaml | helm install app-test .

@AndreCbrera It is not clear to me how that would work with argo? I'm new to this so maybe I'm missing something...
Edit: your exact example confused me, but I see what you're suggesting and I can certainly come up with some such work around (though I don't think this would work, given my example, for sharing the same application.yaml for multiple clusters), but my comment is here to back the concept of having custom environment variables and my example is demonstrating where such a thing would be useful.
Thank you for your suggestion :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet