Environment Secrets are not available on Reusable Workflow / Workflow Templates

[AllanOricil]

Describe the bug

obs: this feature works as designed, but I believe it could be improved.

Problem:
Passing an environment containing secrets to a reusable workflow is not enough to have the environment secrets avaiable.

Example:
In a repository, there is an environment called "myenv", which contains a single secret called "MY_SECRET".
In this repository, there is also a workflow calling a reusable workflow.

This is the reusable workflow

name: Reusable Workflow

on:
    workflow_call:
        inputs:
            ENVIRONMENT:
                required: true
                type: string            
jobs:
    some-job:
        runs-on: ubuntu-latest
        environment: ${{ inputs.ENVIRONMENT }}

        steps:
            - run: env

And this is the workflow

name: Workflow Caller

on:
    push:
        branches:
            - **

jobs:
    some-job:
        uses: Org/repo/.github/workflows/reusable-workflow.yml
        with:
            ENVIRONMENT: myenv

When running this workflow, MY_SECRET isn't available. I see something like this in the logs:

...
MY_SECRET:
...

instead of this

...
MY_SECRET:***
...

In order to make MY_SECRET available in the reusable workflow, I must explicitly write it in the workflow caller, like so:

name: Workflow Caller

on:
    push:
        branches:
            - **

jobs:
    some-job:
        uses: Org/repo/.github/workflows/reusable-workflow.yml
        with:
            ENVIRONMENT: myenv
        secrets:
            MY_SECRET: ${{ secrets.MY_SECRET }}

Why can't the reusable workflow load all of the environment secrets automatically using just the environment's name? Is there a reason for not doing it?

In this repo you can find all of my experiments:
https://github.com/AllanOricil/workflow-template-bug

Expected behavior
"Deployment Environment" secrets should be available in reusable workflows

What's not working?

"Deployment Environment" secrets are not available in reusable workflows

Job Log Output

 ...
    MY_SECRET:
 ...

[AllanOricil]

@ChristopherHX @ericsciple could you help me with this. Im not seeing anything wrong here :/

[AllanOricil]

The funny thing here is that the protection rule works, but the environment variables for the given environment are not available :/

[AllanOricil]

I have updated the bug description with a repro case showing the problem.

[ChristopherHX]

You probably have to pass all environment secrets you want to use in a reusable workflow explicitly. You only set the environment in your called workflow ( which is required, but not enough ), but you didn't explicitly pass environment secrets.
See https://github.saobby.my.eu.orgmunity/t/reusable-workflows-secrets-and-environments/203695/18.

[AllanOricil]

@ChristopherHX Will it get the environment secrets or the repository secrets? It does not seem intuitive to do it like that :/

[AllanOricil]

@jenschelkopf could you help here as well.

[ChristopherHX]

Will it get the environment secrets or the repository secrets?

Try and see, I never tried it.

It does not seem intuitive to do it like that :/

Yeah ask GitHub why. I'm not affiliated with GitHub, microsoft etc.

[MostefaKamalLala]

@AllanOricil I just faced the same problem, it obligates us to have secrets with the environment prefix.
Like newp_SF_ORG_ALIAS, prod_SF_ORG_ALIAS.
We do not leverage the power of environment secrets... I think this issue should reopen.
Do we know if it is tracked?

[AllanOricil]

It is closed because their suggestion works. Check this repo I created that show common mistakes people will make because of the way they implemented it. https://github.com/AllanOricil/workflow-template-bug

[ivellios]

It is closed because their suggestion works. Check this repo I created that show common mistakes people will make because of the way they implemented it. https://github.com/AllanOricil/workflow-template-bug

Shouldn't this line in your code actually point to workflow-template-fix-without-required-secret.yml?

Anyway, your fix is what I've done and it does not work. I still don't get any secrets from the environment.

Also: required secret does not throw error for me.

And I have tried multiple cases:

• env_name in my case replaced with ENVIRONMENT input name
• putting only env secrets into the on expression (because others should be - and are - available anyway)

Nothing helps at this point for me. The only solution I see is to copy-paste the workflow code instead of using reusable workflows.

EDIT:

OK, it works, I had a bug in my code:

      ENVIRONMENT: needs.determine_environment.outputs.env_name

instead of:

      ENVIRONMENT: ${{ needs.determine_environment.outputs.env_name }}

This is still a bizarre solution to use. I agree this should work out of the box only by passing the env information and setting it in the reused workflow.

[brandongallagher-tag]

Can this please be re-opened? This is just extremely inefficient and resulted in me having to move our deployment steps into the release steps workflow file which is unpleasant to look at and couples our entire release pipeline too closely together from a structural point of view.

How Github thought this was acceptable is beyond me.

[MostefaKamalLala]

Can this please be re-opened? This is just extremely inefficient and resulted in me having to move our deployment steps into the release steps workflow file which is unpleasant to look at and couples our entire release pipeline too closely together from a structural point of view.

How Github thought this was acceptable is beyond me.

Honestly I thought GitHub actions would offer every useful features from Azure Pipelines and more.

[austinsasko]

I don’t see it mentioned anywhere, so to add context on this, GitHub implemented this by design and this is identical to the behavior for actions. The called workflow/action can never access environmental things like secrets because actions and reusable workflows are intended to be decoupled from the execution environment. It must be passed into the workflow/action from the callee workflow environment

Although, I do agree it would be much more ideal and developer-centric if this capability was an option rather than a mandate, I wanted to add context in case it helped anyone understand the reasoning for that.

See here: https://docs.github.com/en/enterprise-cloud@latest/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow

[brandongallagher1999]

That's not adding anything to the conversation, and quite frankly, is not helpful in the slightest. That's the point of opening Issues in the first place. @austinsasko

It's already established that GitHub implemented that by design, as that's what our quarrel is with. Thus why we opened this Issue in the first place.

We opened the issue because:

1. Environments are the only way to protect workflows with approval
2. We want to have a deployment/workflow template that is able to be protected by approval.
3. Reusable workflows do not allow for environments and require you to pass secrets into the workflow call, which is unintuitive.
4. You can not protect the reusable workflow from neither the initial call from Workflow A nor inside of reusable Workflow B.

As a result, you are left with the only option of including each deployment job within the first workflow file which:

1. Crowds the workflow file with every single deployment job, which makes for one massive workflow file.
2. Forces you to potentially repeat the same deployment process for X amount of environments.
3. Couples each job too tightly together which stacks tech debt in the future.

For TL;DR:
We need to protect workflows, and their respective secrets while allowing for templates to avoid copy pasta level engineering practices.

[nopol10]

Does secrets: inherit solve this issue? Works for me but I'm not using it with Environments.

https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idsecretsinherit

[brandongallagher1999]

Hi @nopol10 ,

The problem still exists that we can't use environments to protect workflows with approval. That was the main issue that we are concerned about.

Ideally, I'd like to call a reusable workflow and also couple it with workflow approval. So that only certain org/repo members can trigger staging/prod deployment workflows.

[Dgame]

Does secrets: inherit solve this issue? Works for me but I'm not using it with Environments.

https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idsecretsinherit

No I tried to use that and I get

Secret TEST is required, but not provided while calling.

But if I'm not requiring that secret I can use it in the resuable workflow just fine.

[StephenWithPH]

https://github.com/orgs/community/discussions/25238#discussioncomment-3247035 shows a simplified example that works.

[rodrigoaraujo-brasoftwarecloud]

do you have any solution for this case?

[brandongallagher1999]

@bswmservices I suggest combining all jobs on a workflow that require approval into 1 massive workflow file. It's the only solution, unfortunately.

[AllanOricil]

When I say "Environment Variables" I mean this feature:

"Environment Variables" are loaded without passing them in the caller workflow.

Different behaviour for the same thing is kind weird. For example:

Considering:

• an Environment called myenv with a Variable called MY_VARIABLE with value equal to test
• and the following workflows (caller and reusable):

name: Workflow Caller

on:
    push:
        branches:
            - **

jobs:
    some-job:
        uses: Org/repo/.github/workflows/reusable-workflow.yml
        with:
            ENVIRONMENT: myenv

name: Reusable Workflow

on:
    workflow_call:
        inputs:
            ENVIRONMENT:
                required: true
                type: string            
jobs:
    some-job:
        runs-on: ubuntu-latest
        environment: ${{ inputs.ENVIRONMENT }}

        steps:
            - run: echo ${{ vars.MY_VARIABLE }} 
            

This is the output:

test

As you can see, I did not pass Environment "Variables" (vars) from the workflow caller and yet they were loaded in the reusable workflow. Why can't the Environment "Secrets" (secrets) behave the same way?

[tao-zhang-shell]

Thank you, @AllanOricil . I'm seeing exactly the same issue.

[Jangsukwoo]

I think this is bug.

[gyund]

@AllanOricil I think the reason is security related. If Opt-out was the behavior, most people would leave the defaults which would open the footprint for malicious 3rd party actions to steal secrets. Manually requiring secret selection to pass requires more of a manual effort to shoot your own foot.

[AllanOricil]

@gyund I agree with it. It is just not intuitive. I had to spend some time running experiments because the docs were not clear about this behavior.

[AllanOricil]

I also think It is kind weird to bind an expression that isn't resolved in the caller workflow in order to make the secret available in the reusable workflow. So, I would prefer to do something like shown below in order to "opt in".

name: Workflow Caller

on:
    push:
        branches:
            - **

jobs:
    some-job:
        uses: Org/repo/.github/workflows/reusable-workflow.yml
        with:
            ENVIRONMENT: myenv
        secrets:
            - SECRET_ONE
            - SECRET_TWO

[tmehlinger]

@AllanOricil the way you've described it is exactly how it should work. This is a completely absurd limitation in Actions that entirely defeats the purpose of reusable workflows. If a reusable workflow is coupled to identifiers in the calling workflow, why bother writing reusable workflows?

Github, please fix this. If I can opt into passing secrets to a downstream workflow, there is no security issue.

[AllanOricil]

I closed this because their design makes sense. Developers must choose which secrets they want to pass down to reusable workflows since there could exist 3rd party workflows that could send our secrets to services we are not aware of.

[brandongallagher1999]

@AllanOricil Shouldn't be closed. This is a horrible design. Completely restrictive, especially in 2023 when we want to implement basic DRY principles into our DevOps lifecycle. That's what approvals are for, if anyone's concerned about using 3rd party workflows with their secrets.

[xeroc]

Does secrets: inherit solve this issue? Works for me but I'm not using it with Environments.

https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idsecretsinherit

This one weirdly resolved it for me despite using environments.
So the workflow has no knowledge over the environment I use (other than passing a variable), and the re-usable puts and loads the environment, and apparently feeds the correct secrets into ${{secrets*}}.

[samoht9277]

@brandongallagher1999 I completely agree. I have a workflow that calls another one, and I need to pass a string, that the called workflow then resolves as the environment: ${{ inputs.ENVIRONMENT, but when I try to access to environment specific secrets, they don't appear.

[samoht9277]

@xeroc How can I declare the environment key on the caller workflow, when I need to create a new job to call the reusable workflow, and i can't declare the environment key inside that job?

[ben-umble]

@samoht9277

Not sure if this is too late, but may be useful for anyone else struggling.

I've run into this and using secrets: inherit as mentioned above sort of solves the issue. You can't define environment on the caller workflow as such but you can pass it down as an input.

The reusable workflow needs to have an input to receive the environment. Then on your job, you can use environment: ${{ inputs.env }}.

name: Build and deploy

on:
  workflow_call:
    inputs:
      env:
        required: true
        type: string

jobs:
  build:
    runs-on: 'ubuntu-latest'
    environment: ${{ inputs.env }}

On the caller workflow, pass down the environment and declare secrets: inherits like this:

jobs:
  deploy:
    uses: org/repo/.github/workflows/build.yml@main
    with:
      env: Development
    secrets: inherit

The only thing I'm not clear on is if this passes the secrets down further into third-party workflows so treat with caution.

[sap147]

Does secrets: inherit solve this issue? Works for me but I'm not using it with Environments.
https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idsecretsinherit

This one weirdly resolved it for me despite using environments. So the workflow has no knowledge over the environment I use (other than passing a variable), and the re-usable puts and loads the environment, and apparently feeds the correct secrets into ${{secrets*}}.

@xeroc - Can you please elaborate how you are feeding secrets into ${{secrets*}} in the reusable workflow? Thanks.

[wkhatch]

@sap147 secrets: inherit solved the issue for me. For completeness:

jobs:
  deploy-development:
    uses: ./.github/workflows/reusable_workflow.yml
    with:
      environment: development
    secrets: inherit

and in the reusable:

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
        <whatever you expect to be passed via the inherit, so list each one. maybe don't have to do this, but it can't hurt and it works for me>

and then elsewhere in the reusable workflow where you want to reference a secret, it's the normal syntax ${{ secrets.SOME_SECRET}}

CRITICAL UPDATE: This actually doesn't work, when trying to access secrets defined in and scoped to an environment. Originally, it seemed to work, but, I hadn't removed repository level secrets that had the same name. Once I realized that, and fixed it, the workflow immediately errors on run, with no logs or anything. Crazy how janky this whole thing is. I'm going back to circleci

[obaqueiro]

I think this issue should be reopened. We are hitting the same issue which prevents us from using the "reusable workflows".

In our case, the problem we have is that the keyword environment under jobs.job-name is not accepted when using the uses: key. So, our Github configured secrets (in different environments) are not visible to the called yml workflow. We need either a way to define an environment (where secrets are pulled from ) at the workflow level , or to be able to use the environment key within a job that calls a sub-workflow witht he uses: keyword.

[michealroberts]

I am also seeing this issue, anything in the "Environment Secrets" section of my repository are coming through as empty in the workflow runner ...

[connorads]

Well that's a few hours I won't get back 😅

The secrets: inherit "workaround" is working for me.

FYI @wkhatch, regarding your CRITICAL UPDATE: I don't have any overlapping names in repo/org secret, SOME_ENVIRONMENT_SECRET is only defined in environments. But it still works.

# calling-workflow.yml
jobs:
  example-dev:
    uses: ./.github/workflows/reusable_workflow.yml
    with:
      env: dev
    secrets: inherit

# reusable_workflow.yml
on:
  workflow_call:
    inputs:
      env:
        required: true
        type: string
    secrets:
      SOME_REPOSITORY_SECRET:
        required: true
      SOME_ENVIRONMENT_SECRET:
        required: true

jobs:
  example:
    name: Example
    runs-on: ubuntu-latest
    environment: ${{ inputs.env }}
    steps:
      - run: echo "${{ secrets.SOME_REPOSITORY_SECRET }}"
      - run: echo "${{ secrets.SOME_ENVIRONMENT_SECRET }}"

[wkhatch]

Hmmm, very strange. That’s how I’d originally structured my workflows as well. I’ve since given up on environments and just add branch dependencies, and create separate variables and jobs for each. Bill On Dec 29, 2023, at 1:14 PM, Connor Adams ***@***.***> wrote: Well that's a few hours I won't get back 😅 The secrets: inherit "workaround" is working for me. FYI @wkhatch, regarding your CRITICAL UPDATE: I don't have any overlapping names in repo/org secret, SOME_ENVIRONMENT_SECRET is only defined in environments. But it still works. # calling-workflow.yml jobs: example-dev: uses: ./.github/workflows/reusable_workflow.yml with: env: dev secrets: inherit # reusable_workflow.yml on: workflow_call: inputs: env: required: true type: string secrets: SOME_REPOSITORY_SECRET: required: true SOME_ENVIRONMENT_SECRET: required: true jobs: example: name: Example runs-on: ubuntu-latest environment: ${{ inputs.env }} steps: - run: echo "${{ secrets.SOME_REPOSITORY_SECRET }}" - run: echo "${{ secrets.SOME_ENVIRONMENT_SECRET }}" —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[funkswing]

The workaround presented here is limited to the reusable workflow being in the same repository as the caller workflow and/or the caller workflow having the Environment with the secret defined.

That greatly limits the reusable workflow. For example, if we want to define a reusable workflow owned by an ops dev in its own repo and lock down usage of an Environment secret defined only in the repo, then this does not work. You can't call that workflow from other repos and have access to that secret. This may be by design, but makes managing sensitive Environment secrets at scale extremely burdensome (to the point where Org secrets and Environment secrets are not valid solutions to sensitive secrets).

[AnthonyGress]

@samoht9277

Not sure if this is too late, but may be useful for anyone else struggling.

I've run into this and using secrets: inherit as mentioned above sort of solves the issue. You can't define environment on the caller workflow as such but you can pass it down as an input.

The reusable workflow needs to have an input to receive the environment. Then on your job, you can use environment: ${{ inputs.env }}.

name: Build and deploy

on:
  workflow_call:
    inputs:
      env:
        required: true
        type: string

jobs:
  build:
    runs-on: 'ubuntu-latest'
    environment: ${{ inputs.env }}

On the caller workflow, pass down the environment and declare secrets: inherits like this:

jobs:
  deploy:
    uses: org/repo/.github/workflows/build.yml@main
    with:
      env: Development
    secrets: inherit

The only thing I'm not clear on is if this passes the secrets down further into third-party workflows so treat with caution.

This solved it for me, thank you

[pmoros]

@ben-umble Thank you so much!

[coreyd-valcre]

The workaround presented here is limited to the reusable workflow being in the same repository as the caller workflow and/or the caller workflow having the Environment with the secret defined.

That greatly limits the reusable workflow. For example, if we want to define a reusable workflow owned by an ops dev in its own repo and lock down usage of an Environment secret defined only in the repo, then this does not work. You can't call that workflow from other repos and have access to that secret. This may be by design, but makes managing sensitive Environment secrets at scale extremely burdensome (to the point where Org secrets and Environment secrets are not valid solutions to sensitive secrets).

I have to agree. I am utilizing a different repo which holds all of my re-usable workflows and actions and this is just a pain to manage. If my vars and secrets are repository based or org based I have no issue pulling them, but environment based is a whole different conversion. I should be able to set my Environment in my workflow where I am calling so it can be passed down and still access the values. This makes having re-usable workflows pretty much worthless since I still need to copy this dozens of times.