AWS credentials are not passed to some nix commands

[ingun37]

I use S3 as the binary cache. I configure AWS credential using configure-aws-credentials. This action internally sets the system environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. These env vars are used by all the AWS operations throughout the CI/CD.

Problem is that some nix commands recognizes the env vars and some don't depending on whether they are ran behalf of nix daemon or not (The inspection is detailed in NixOS/nix#5723). For example, nix path-info -r /nix/store/... --store s3://some-bucket would work while nix-build would not.

I fixed it by adding

    - uses: cachix/install-nix-action@v31
      with:
        install_options: --no-daemon

but I wish I don't have to do this.

Can it be fixed at the install-nix-action level?

[sandydoo]

This is generally out of scope for us, but we could try implementing something if you have an idea how to do it.

The only thing that comes to mind is we could try setting access-tokens in nix.conf like we do for the GitHub token. I don't know if this is possible.

[secobarbital]

@ingun37 can you say more about the --no-daemon fix? Did you still have to write the credentials into a file somewhere to make it work in GitHub Actions? I am still getting the error despite adding that option:

warning: AWS error fetching 'nix-cache-info': The specified bucket does not exist (request id: XXXXXXXXX)

[ingun37]

I remember fixing that warning by granting ACL to the role. I can answer better only tomorrow (because i need to go the office to access private networks) but in the mean time you can see what it says by giving --verbose=4. It logs every requests and responses.

[arianvp]

This is the fix:

The problem is that substitution happens in the nix-daemon. So the nix-daemon needs access to the credentials

I don't think this is anything this action can / has to fix.

    - uses: cachix/install-nix-action@v31
    - name: Assume AWS Role
      uses: aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722A # v4.1.0
      with:
        aws-region: "${{ inputs.aws-region }}"
        role-to-assume: "${{ inputs.role-to-assume }}"
        role-session-name: "${{ github.run_id }}"
        role-duration-seconds: "${{ inputs.role-duration-seconds }}"
    - name: Make AWS Credentials accessible to nix-daemon
      run: |
        sudo -i aws configure set aws_access_key_id "${AWS_ACCESS_KEY_ID}"
        sudo -i aws configure set aws_secret_access_key "${AWS_SECRET_ACCESS_KEY}"
        sudo -i aws configure set aws_session_token "${AWS_SESSION_TOKEN}"
        sudo -i aws configure set region "${AWS_REGION}"

[secobarbital]

Thank you so much!

[Mic92]

In the next nix version we will also support aws sts NixOS/nix#12646
in case this helps somehow for this issue.

[arianvp]

It does not. This is nothing AWS specific. It's that the nix daemon gets its credentials from its environment and not the environment nix command runs in

[sandydoo]

@arianvp, do you mind if I add a modified version of your example to the FAQ?

[ingun37]

You can close the issue if it's not the action's job.

[arianvp]

Not at all! Please do. I waive any copyright I have over this example. (Probably small enough that it doesn't apply in the first place). Feel free to add