dynamic runtime environment variables

[janesser]

While striving for immutable build artifacts we came across some possible improvement points.

First let me explain the motivation behind "immutable build artifact". Alongside of the stages like e,g, DEV, TEST, QA, UAT, PROD the configuration changes, as an example nuxt-axios API_URL and API_BROWSER_URL. Ideally this can be injected by env vars, which pretty much fits the kubernetes way of dealing with configuration.

Presently a work-around can be achieved with help of https://github.com/samtgarson/nuxt-env, which does a decent job in providing needed injection mechanism. The down-side is that no nuxt-plugin by default goes for this.$env, which drives one in a lot of customizing of plugins. Not to mention the broken upgrade ease.

Can you imagine a way to support this natively in some of the next nuxt releases?

_{This question is available on Nuxt community (#c8719)}

[pi0]

Hi @janesser. I've transferred issue from RFC to here because it is more an enhancement discussion.

Values passed to the env section of nuxt.config will be replaced by a webpack plugin during the build. Would you please elaborate more what you expect from storing it in build artifact?

[vhf]

I also ran into this problem, here is one of my example case:

I'm using nuxtjs auth module with some OAuth API. Here is what the config looks like: https://auth.nuxtjs.org/reference/providers/github , in nuxt.config.js:

auth: {
  strategies: {
      github: {
        client_id: '...',

I'd like to build my app once and deploy it many times with different configuration values. This cannot be done because client_id: process.env.FOO_CLIENT_ID will be replaced at build time and the build-time value will be written in the build artifacts.

Here you couldn't build your app in a container and deploy it with different FOO_CLIENT_ID values, you would have to build the app each time you start a container.

I hope this makes the benefits of having nuxt projects configurable at runtime a bit clearer.

[aldarund]

@pi0 its more about have runtime env vars like in the module that is linked. E.g. build once deploy to different environments same artifact, isntead of building each time for each env ( e.g. staging, test, production etc )

[pi0]

I see the point. And auth and axios are probably best examples for such need. However, this is a duplicate topic and issue was discussed here is a short brief of status:

• 1. process.env.* from server side is runtime configurable. axios module supports it for example for API_URL
• 2. process.env.* from client-side when running in SSR mode, is only changeable if we somehow pass it to nuxtState. The simplest way would be using a store module.
• 3. process.env.* from client-side when running in SPA or Static generated mode, is only changeable by doing an API call.

Having built-in support for case (2) could be possible but potentially opens a way to critical security holes and is only usable for SSR users.

Regarding case (3) also could be possible to store app config in a json file inside static dir and make client to download it on each visit. Has overhead but at least makes it somehow possible.

[janesser]

@pi0 can you link the original discussion that you refer too, please?

@vhf @aldarund thanks for supporting the need.

regarding the security, i am confident that docker envvars are sufficiently safe.
having something in javascript either hard-coded or dynamically refreshed is also in the same security class. Maybe i miss some attack vectors.

i felt like you hit the sensible point where it came to the way that webpack4 build routine is instrumented. Probably the present behaviour is at least partially inherited.

regarding possible solutions, did you check nuxt-env? by my experience it covers SSR plus non-SSR in a consistent way, not sure how it is solved internally. //cc @samtgarson
EDIT: it uses nuxtState around here: https://github.com/samtgarson/nuxt-env/blob/master/lib/plugin.js

ill try to collect some insight on webpack4 and eventually read through the internals of nuxt-env to give you some additional pointers.

[samtgarson]

@janesser in the meantime open an issue for any confusing things you're hitting—I picked $env as it seemed appropriate but we can look at changing it or allowing it to be dynamic pretty easily.

@pi0 all I do is use a middleware to add the process.env values onto the req object, then a plugin which uses nuxtState to read from req and inject them onto the context.

Would be great to have this natively, it's currently impossible to use Nuxt in any kind of containerised, multi-environment setup with the existing native setup, or at least anywhere where the production environment could vary from the build environment.

[samtgarson]

(non SSR and SPA are still scenarios that I don't think nuxt-env covers super smoothly, but it's OK...)

[pi0]

Mainly discussed in #618.

We are aware of @samtgarson nuxt-env project. That's implementing (2) method which would be useful for users that want it. It is more than qualified to be a nuxt-community project 👍

About security matter. Docker can't prevent one accidentally passing client_secrect to the user's browser window. It is the user's responsibility to be aware and ensure that.

/cc @manniL @aldarund

[janesser]

About secrets retention, check this one: samtgarson/nuxt-env#9

To wrap up, is there a way to make process.env as consumed by default from nuxt-plugins in the same way as process.$env?

[samtgarson]

regarding secret env vars: nuxt-env supports these as of this PR but as @pi0 says, it's still up to the developer to ensure these don't get passed to the client.

Would be keen to help any efforts of integrating this or moving it over to nuxt-community, let me know if either of those are desired.

[axetroy]

This is my solution:

set dynamic runtime environment variables in store

so we can get it in client side and server side via store.state.env.

// store/index.js

export const state = () => ({
  env: 'development'
})

export const actions {
  // run in every SSR
  async nuxtServerInit (store, context) {
    // read runtime environment everytimes and set to store
    const env = process.env.NODE_ENV || 'development'
    store.commit('SET_ENV', env)
  }
}

export const mutations = {
  SET_ENV (state, env) {
    state.env = env
  }
}

Use in middleware

// middleware/env.js

export default function ({store}) {
  console.log(store.state.env)
}

Use in page/layout/component

<template>
  <div>{{env}}</div>
</template>

<script>
export default {
  data () {
    return {
      env: this.$store.state.env
    }
  }
}
</script>

[janesser]

@pi0 @samtgarson what's the next step here?

[vhf]

I see the point. And auth and axios are probably best examples for such need.

Definitely. The ability to configure auth, axios and apollo at runtime would be immensely helpful to people who wish to build once and deploy many times.

I'd be happy to contribute if the core team agrees on what the right approach should be.

[fvigotti]

** EDITED to add some elements for clarification of the issue

the problem is that process.env.* gets replaced with "live(build-time)" placeholders during build time,
so runtime env variables that are needed for a 2019-well-designed app to contain things such "axios/apollo links -> which are also needed during those plugins configuration ( in nuxt.config.ts ) " are broken, the solution is https://github.com/samtgarson/nuxt-env and reconfigure those plugins using a plugin in "plugins/..." which use the new injected runtime env from ctx.app.$env there ( the official docs of https://github.com/samtgarson/nuxt-env doens't talk about plugins usage , just store/component )

anyway, I'm quite new with nuxt/vue and having to know so much internals to runtime-configure the application and having to read the (0-comments) source code of https://github.com/samtgarson/nuxt-env to figure out how to configure apollo has been a pain...

my 2 cents are that this is a serious issue, I've lost 3 hours trying to figure out why my local dev env works and my deploy env didn't , the problem was that there was two different graph of injected variables because the build process interpolated them from different build environment,

I think that saying that there is no official way -to use env variables || to inject configuration- at runtime but that https://github.com/samtgarson/nuxt-env that "works" , or to use env variables at build-time (which is an huge anti-pattern in 2019 ) must be written at least here https://nuxtjs.org/api/configuration-env/ with bold characters untill resolved.. and maybe propose this module https://github.com/samtgarson/nuxt-env as a solution ? anyway having someting so hardly testable, so important (that is used to pass auth / tokens / configuration to the application ) on a non-official module (with all the eventual future compatibility problem, version-pairing, .. ) is a sign of project immaturity and should be incorporated in the main app asap.

I want to add that this is a very subtle issue, because the nuxt.config.(ts|js) is run twice, at build time and at runtime, but at build time it will compile apollo/axios and other modules with env variable that are present at build-time , when you run at build time, and try to debug, the nuxt.config.js file is executed with correct env variables, but the configuration for the module that you expect to gets configured during this execution are already configured with hardcoded build-time-value...

[samtgarson]

@pi0 I'm happy to move the module over to nuxt community if that would help the situation, let me know

[TheAlexLichter]

@samtgarson Let's discuss that on https://discord.nuxtjs.org/ ☺️

[rmeissn]

Just to give some idea of a possible solution:
We had the same issue in my old project and we splitted the webpack build step into two steps. One, that builds most parts of the application (e.g. executed by CI/CD) and one smaller step that updates a file with predefined env vars before starting the application. See scripts build:nostart and start in package.json

So the basic steps for a docker deployment (on container execution) are:

1. An entrypoint script is executed that uses (in our case) envsubst and a template file to create a config file, containing predefined env vars (so they are hardcoded to the resulting json file)
2. Run a limited webpack build step in order to include this newly created file into the already existing distribution
3. Execute the distribution (that is now using the env vars used on container startup)

I can perfectly image that these steps may be included into nuxt start command.

[tomsaleeba]

I have (had) a need for this too and I use nuxt-env (excellent module) where I can but there are places that nuxt-env can't help like configuring your API URL for the nuxt-proxy module in the nuxt.config.js file.

I run in Docker so I decided to get a bit hacky and just use placeholders during the docker build:

# Dockerfile
...some stuff

RUN \
  export KEYCLOAK_CLIENT_ID='%%KEYCLOAK_CLIENT_ID%%' && \
  export KEYCLOAK_HOST='%%KEYCLOAK_HOST%%' && \
  export KEYCLOAK_REALM='%%KEYCLOAK_REALM%%' && \
  export API_BASE_URL='%%API_BASE_URL%%' && \
  yarn build

... more stuff

Then in the entrypoint.sh for the container, just parse the build JS files:

# entrypoint.sh
find .nuxt/ \
  -type f \
  -name '*.js' \
  -exec sed -i "s+%%KEYCLOAK_CLIENT_ID%%+${KEYCLOAK_CLIENT_ID:?}+g" '{}' \; \
  -exec sed -i "s+%%KEYCLOAK_HOST%%+${KEYCLOAK_HOST:?}+g" '{}' \; \
  -exec sed -i "s+%%KEYCLOAK_REALM%%+${KEYCLOAK_REALM:?}+g" '{}' \; \
  -exec sed -i "s+%%API_BASE_URL%%+${API_BASE_URL:?}+g" '{}' \;

exec yarn start

It's not glamorous but it gets the job done. It works for local dev (yarn dev) too because you can pass the env var directly. The downside is that you have to update the docker files when you need a new var but I can live with that. You could always get fancy and scan the env in the entrypoint for a certain prefix and then update all the matching env vars, but that's gold plating for me right now.

[stale]

Thanks for your contribution to Nuxt.js!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you would like this issue to remain open:

1. Verify that you can still reproduce the issue in the latest version of nuxt-edge
2. Comment the steps to reproduce it

Issues that are labeled as 🕐Pending will not be automatically marked as stale.

[vhf]

This is still an important issue for those who'd like to build easily deployable products with Nuxt. It would be good if stalebot didn't close it IMO.

[pi0]

Hi @joaquimds you can now try RuntimeConfig which also supports SPA (no need to inject from app.html)

Closing the issue as it is landed to 2.13.0.

Thanks again @janesser @samtgarson and all feedbacks on this issue ❤️

Do you know if there are any plans to help module authors in the community (e.g. axios module) to adopt these new configuration options? That would be the last step to help get these issues sorted for good.

Will be landed soon on http/axios modules

[blowsie]

@pi0 RuntimeConfig is great, but it doesnt allow use to use environment variables in our nuxt.config.js in a dynamic way. Think one build, with different configuration for Test/Staging/Prod

Is there an open ticket for this problem? Should this one be re-opened?

[SkyaTura]

🤔 I'm not sure if I got your point... could you elaborate it?

(I guess another issue would be better, since the main subject of this one was in fact solved)

[cageyv]

@blowsie i used a dirty sed solution in my last project with old nuxt.js version 2.11.0.
I never try nuxt after 2.13 release, but someone recommend me nuxt-edge package for this problem.
So if you have time please check with nuxt-edge

And i think i know what you want..
You want to solve this downside: "Values are read during build time and persisted into webpack bundle. So for a change to process.env we need to rebuild"
I had the same problem. I wanted to build the container image once and then submit parameters to it at startup.
In the end, this problem was solved by a dirty sed trick.
When I tried to find advice on the net, many came to the same decision.
So I had to leave this option, since there was no more time.

[blowsie]

@cageyv , im already using 2.13 with the runtime config, but it doesnt help the fact when you use process.env in your nuxt.config the variables are translated to strings at build time.

@SkyaTura does this answer your question?

[blowsie]

I guess another issue would be better, since the main subject of this one was in fact solved

Sadly this is not true, the main subject of this was immutable build artifacts, we dont have that yet, at least not in the nuxt.config

[samtgarson]

@blowsie are you using SPA mode or Universal mode?

I'm not as familiar with SPA mode enough to suggest a fix for this, but if you're using Universal mode (i.e. SSR) then using process.env should be run at runtime as well—the only issue from here are 3rd party modules which write their configuration to file at build time (like the Axios module).

For these, I think it would be great if the Nuxt team could help nudge the major in the right direction to support runtime config, I know when people used Nuxt-env this was a common frustration.

[blowsie]

@samtgarson I am indeed using universal mode.

But you are right, the issues im having are with the nuxt modules, like for example nuxt-community/auth-module : nuxt-community/auth-module#713 (comment)

[samtgarson]

Yep. I don't think there's much Nuxt can do while those modules are writing config to file other than help module authors migrate to this strategy.

[clairegraham]

I just ran into this issue (our site runs in SSR mode, on a serverless platform).

My use case:
Our production site uses a different Google Tag Manager ID from the staging site. So, I still deploy the site to both environments in a "production" node environment, but they have various configuration differences based on where they end up being deployed.

I tried to use nuxt-env (which is a great module!) but it wasn't accessible via nuxt.config.js, which is where the GTM module configuration resides in https://github.com/nuxt-community/gtm-module. This was my solution:

package.json:

{
  "scripts": {
    "deploy:staging": "DEPLOY_TARGET=staging nuxt build"
    "deploy:prod": "DEPLOY_TARGET=production nuxt build"
  }
}

nuxt.config.js

module.exports = {
  gtm: {
    id: process.env.DEPLOY_TARGET == 'staging' ? 'GTM-STAGING-ID' : 'GTM-PROD-ID'
  }
}

[pi0]

Hi @blowsie indeed as @samtgarson mentioned, it is up to module authors supporting runtime config which auth module still hasn't.

@clairegraham You can try latest version of gtm-module. It supports runtime config: https://github.com/nuxt-community/gtm-module#runtime-config

export default {
  modules: [
    '@nuxtjs/gtm'
  ],

  publicRuntimeConfig: {
    gtm: {
      id: process.env.DEPLOY_TARGET == 'staging' ? 'GTM-STAGING-ID' : 'GTM-PROD-ID'
    }
  },
}

[ababbdev]

Hi,
Thank you for doing the work to support the dynamic env values!
We have been migrating to use this strategy however I just want to confirm if this will be the strategy that is carried over to the next version of Nuxt when Vue 3 is added or these are up for changes?

[rotemrevivo91]

it's still not working and not documented anywhere. It's a shame that something important like this get tossed aside for over a year already

[memic84]

Doesn't this solve your problem?
https://nuxtjs.org/blog/moving-from-nuxtjs-dotenv-to-runtime-config
https://nuxtjs.org/docs/2.x/configuration-glossary/configuration-runtime-config

[rotemrevivo91]

No, unfortunately. The problem is that i have a single build (i.e 1 docker image) that is used on multiple environments. Each user should be able to specify their own router base as an .env variable and just run the container without the need to rebuild the image.
Nuxt resources are then can't be found (nuxt .js files).
I was able to use (SSR mode):
nuxt.config.js

  publicRuntimeConfig: {
    baseURLSubDirectory: process.env.BASE_URL_SUB_DIRECTORY,
  },
  router: {
    base: process.env.BASE_URL_SUB_DIRECTORY
  },
  plugins: [
    {
      src: '@/plugins/cdn.js',
      ssr: false
    },
  ],
  hooks: {
    /* Nuxt doesn't allow the publicPath (CDN url) to be overridden at run time - it's baked into the manifest at
    ** build time.  This hook intercepts the VueRenderer when it has loaded the manifest and updates the publicPath
    */ to the current env value.
    render: {
      resourcesLoaded(resources) {
        const path = `${process.env.BASE_URL_SUB_DIRECTORY}_nuxt/`
        resources.clientManifest && (resources.clientManifest.publicPath = path)
        resources.modernManifest && (resources.modernManifest.publicPath = path)
        resources.serverManifest && (resources.serverManifest.publicPath = path)
      }
    }
  }

plugins\cdn.js:

export default function({$config}) {
  if (process.client) {
    __webpack_public_path__ = `${$config.baseURLSubDirectory}_nuxt/`
  }
}

which replaces all resources on run-time! which is important but still i get a 404 page even though nuxt is able to find all the needed resources.

[pi0]

@rotemrevivo91 Would you please open a feature request for supporting runtime baseURL?

[rotemrevivo91]

@pi0 will do :)

[alexcroox]

Did anyone find a solution to supporting multiple environments? The Nuxt docs seem to assume that everyone is running production configs locally or at best only has dev + production. If you have local/sandbox/staging/production this runtime config doesn't seem suitable for large applications with multiple environments before production.

Just getting started with Nuxt, I assume I can use .env files instead.

[samtgarson]

I've used it to run many dynamic environments (we ran a hand-rolled version of Vercel'a preview environments with a deployment for each branch as well as production).

I'd suggest you open a Discussion on the nuxt repo with your specific issues, as (apart from specific pieces of config like above) runtime config should allow you to have different config values per environment.

As mentioned above, many community modules still need to adopt this feature 😕

[rotemrevivo91]

We opened a new feature request #8509 (comment)
The nuxt team will work on way to make it easier but we also found a work around. If you need extra help don't hesitate to ask :)

[pi0]

Locking issue to avoid confusions. Runtime config for multiple envs is supported. Requested feature is for setting baseURL (router.base) after build. If something is not working as expected or need another feature please use new issue.