-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for config files #1584
Comments
Commander does not explicitly track whether an option has been specified on the command-line so I can see that applying the priorities for adding in values from config files is currently tricky. Some of the same issues come up with mixing in environment variables. One work-around approach might be to not specify the default value to Commander, and instead modify the Help to display the external default value (in your example, from |
A couple of thoughts. Regarding proposed API. I think there would be support at the Command level so a startup file can be read without a corresponding option. I'm not sure at the moment if also makes sense as an Option method, or whether handled by client using the Command support. Support for subcommands requires some thought. I think of config files as being global normally. There is probably a low chance of reuse of option names between program and subcommand, but a higher chance of potentially conflicting reuse between subcommands. So for a full solution the config options may need to be qualified by sub-command path if they can be imported globally. |
To implement environment variable support with the desired balance of priority for default/env/cli, I have a PR which uses a new property to add and consult the "source" of the option value. I had config file in mind as another priority level when I was writing it. (Currently private property in the PR: #1587 )
|
To whom it may concern, this is how I currently manage configs. const config = require('./path/to/config')
const useConfig = process.argv.find((arg) => ['--help', '-h', '-d', '--use-defaults'].includes(arg));
const defaults = useConfig ? config : {}
program
.option('-i, --install-dir <dir>'), 'location', defaults.installDir
.option('-d, --use-defaults'), 'silent install', false
.action(async options => {
/** iterate over options and prompt missing fields */
})
... As for manually specifying a config file, would it be possible to do something like const softRequire = path => {try {return require(path)} catch(e){}}
const configPath = getSingleOption(process.argv)('-c --config') || 'mylib.config.js' //made up commander helper
const config = softRequire(configPath)}
const defaults = { /** options */}
const options = { ...defaults, ...config }
program
.option('-c --config', 'config location') // dummy option. Only included for `--help`
.option('-i --install-dir', 'install location', options.installDir)
... EDIT: updated bottom script |
Would love to see this feature in commander. It's currently pretty complicated to implement defaults, a config file, and the ability to override the config file via command line args. |
Assuming config can use a dictionary (hash) of option config values, what keys should be supported? The mapping from dash-separated to dashSeparated is a programmer detail, so I am thinking at least the flags themselves should be supported since these are what end-users work with. program.option('-p, --port-number <number>');
For keys that do not start with a dash, Commander could perhaps look for a long option flag that matches (e.g.
|
I think it would be best to only allow the JavaScript style names of options for keys of the config file. This would mitigate conflicts, and users are familiar with JavaScript style property names in JSON. I bet it would be possible for commander to generate a JSON Schema file for the config file. |
Yes, I support @alexweininger -- supporting only JavaScript style will be enough, at least in the beginning |
What would the equivalent of this be with a builtin config path option? // softRequire is like require, but doesn't throw error on missing files
const softRequire = path => {try {return require(path)} catch(e){}}
// made up commander helper
const configPath = getSingleOption(process.argv)('-c --config') || 'mylib.config.js'
const options = softRequire(configPath)} || {}
program
.option('-c --config', 'config location') // dummy option. Only included for `--help`
.option('-i --install-dir', 'install location', options.installDir)
... I'm a little on the fence about a native configs feature.
My preferred solution would be a const { loadConfig, program } = require('commander')
const options = loadConfig('-c --config', './my-default-config-path.js')
program
.option('-c --config', 'config location') // dummy option. Only included for `--help`
.option('-i --install-dir', 'install location', options.installDir)
... |
Actually, I'm still thinking about lower-level details and don't have a builtin config path option in my head! So this code is a bit different, but where my thinking is up to:
|
If there was an easier way to discern if an option has been passed in by the user or not, implementing a config file while using commander becomes a lot easier. I think solving this problem first might be more of a priority than to create a fully integrated config file feature. Quoting from #1584 (comment)
|
Hopefully solved that problem internally while implementing |
@shadowspawn I like your example, but how would it handle this scenario? //mycli.js
const { loadConfig, program } = require('commander')
// specify default config
const env = process.env.NODE_ENV || 'development'
const defaultConfigPath = `./config/${env}.config.js`
// use user provided config or fallback to defaultConfigPath
const options = loadConfig('-c --config', defaultConfigPath )
program
.option('-c --config', 'config location') // dummy option. Only included for `--help`
.option('-i --install-dir', 'install location', options.installDir)
.option('-a --architecture <arch>', 'system architecture', 'hardcoded-value')
... Expectations:
Note for anyone who thinks configs shouldn't be treated as defaults: Please consider that these are the default values that options will default to. Showing any other value would be misleading. |
Just so we're looking at the whole problem at once, here is a list of all of the places that config information might come from in a full-featured CLI, in order of increasing precedence. Everywhere I say FILE, i mean myCli.js, myCli.cjs, myCli.mjs, or myCli.json, and there might be a built-in mechanism to extend configs contained in other packages:
For each bit of config found, it needs to be carefully combined with the existing set, depending on type. Simple types like bool, string, and choices overwrite. Counts are specified as Note that a program might want to opt out of or into any of these (except the command line, I guess. :) ). |
I was not imagining an overlap between defaults and a configuration file in that way. Interesting though, and clear outline of scenario. I think of the configuration file(s) being outside the help, with perhaps a way to view the details:
Are there some (well-known) programs or frameworks that behave the way you describe?
Two issues:
|
Wide coverage, thanks.
Another custom place is a key of
yaml format has come up in previous discussions (#151), probably outside familiar javascript tooling examples (like
I did see recently that mocha does this with options which can safely be repeated: https://mochajs.org/#merging With environment variables ( |
+1 to package.json. I think it goes between 2 & 3. +1 to yaml being possible. Allowing for custom parsers is better than taking a dependency to do yaml out of the box, imo. I don't think that env variables should always change the help to show a different default. If you go out of your way like @jakobrosenberg did in his example, you can work around that somewhat. Also, in that example, if your config file is .js, you can do the NODE_ENV check in there and just have one file. I get that only solves that one particular case, but it's something to remember; we don't have to provide ultimate flexibility in places where the user can use JS to customize. |
One thing to keep in mind through all this is separation of responsibilities so we don't end up with a Frankenstein's monster or Homer's car. There are numerous config resolvers that could easily be used with Commander. All you need is just the path of the config file, which could be solved with a simple helper that returns the value of a single argument, eg. Adding a small helper vs a full-blown config resolver would add
|
If we could expose this so that we can consume this in the same way you did when implementing |
Lots of good comments. Selected highlights:
@alexweininger #1584 (comment)
@alexweininger #1584 (comment)
@jakobrosenberg #1584 (comment)
The two things I am currently thinking about are:
For now, I am thinking of leaving it up to user to locate desired implicit config file (or |
Both of these sound great to me. And I totally agree with:
|
(This is to find config file from arguments, before doing full parse and help.) I see the attraction for you to allow using the config as defaults so they are displayed in help, as in your example. The full parse does recursive processing of subcommands with a lot of complexity, both for processing valid syntax and for showing good error messages. My instinct is that while it may be possible to robustly extract an option value for a single program where you know the syntax and configuration you are using, it would be hard to do robustly for a general solution. |
@shadowspawn for Commander to imports configs, that would be my favorite, yes. But I would still prefer that Commander handles arguments parsing and leaves module importing to native tools like const configPath = getOption('-c --config')
const config = require(configPath) Having a // we want to include the defaults if we're doing a silent install or if we're printing the help page
const useDefaults = getOption('-s --silent') || getOption('-h --help')
const defaults = useDefaults ? require('./config.js') : {}
program
.option('-i, --install-dir <dir>'), 'location', defaults.installDir
.option('-s --silent', 'install using default values')
.action( async options => {
// if we didn't use `--install-dir` or `--silent`, installDir is undefined and we will prompt the user
options.installDir = options.installDir || await promptInstallDir()
}) You could even combine the above example with config import. That would allow us to obtain config values by example// we want to include the defaults if we're doing a silent install or if we're printing the help page
const useDefaults = getOption('-s --silent') || getOption('-h --help')
const defaults = useDefaults ? require('./config.js') : {}
// load config
const configPath = getOption('-c --config')
const config = require(configPath)
Object.assign(defaults, config)
program
.option('-i, --install-dir <dir>'), 'location', defaults.installDir
.option('-s --silent', 'install using default values')
.action( async options => {
// if we didn't use `--install-dir` or `--silent`, installDir is undefined and we will prompt the user
options.installDir = options.installDir || await promptInstallDir()
}) |
It was more complicated enforcing the syntax than I expected, but I have some proof of concept code working. Example usage: const { program, Option } = require('commander');
program
.option('-d, --debug')
.option('-c, --color')
.option('-C, --no-color')
.option('-o, --optional [value]')
.option('-r, --required <value>', 'string', 'default string')
.option('-f, --float <number>', 'number', parseFloat)
.option('-f', 'number', parseFloat)
.option('-l, --list <values...>')
.addOption(new Option('-p, --port <number>').env('PORT'))
.option('--use-config')
program.parse();
if (program.opts().useConfig) {
const config = {
debug: true,
color: true, // same as CLI --color
color: false, // same as CLI --no-color
'--no-color': true, // same as CLI --no-color
'-o': true, // same as CLI -o
'optional': 'config string',
'required': 'config string',
'float': '3.4',
'list': ['apple', 'pear'],
'port': '443'
};
program.mergeConfig(config);
}
console.log(program.opts());
(Edit: fixed priority order of env and config) |
Is it best to display a merge error if a config key does not match a declared option? Displaying an error is helpful for picking up mistakes early, but prevents directly using a flat configuration file across multiple subcommands with mixture of unique options (by ignoring unknowns). |
I think, you can return the error from
Wouldn't it be better to name key as |
|
FYI @jakobrosenberg And came up with something along the lines you were suggesting:
|
@shadowspawn , sorry completely missed your previous message. Love the example. - Though the second parameter (name) should probably be derived from flags. function getSingleOption(flags, args) {
// flagToCamelCase for lack of better words ^^
const name = flagToCamelCase(flags)
...
} Regarding the PR, I can't fully wrap my head around it, but I assume it abstracts the commented logic. /** resolve defaults, configs and env */
...
const defaults = { ...myDefaults, ...myConfig, ...myEnv }
program
.option('--cheese [type]', 'cheese type', defaults.cheese) |
I just wanted to voice support for a lighter touch approach, rather than Commander being responsible for locating/opening/parsing config files itself. There's already fantastic tools that make this easy (e.g. cosmiconfig). If Commander exposes a way to detect an option's source (or even a way to merge an arbitrary object into the options), then those existing tools can be leveraged easily, while still leaving the door open to whatever more niche approaches a given project might need. |
Did a deep dive on config files, but for now just added |
@shadowspawn Sorry for resurrecting this, but I have a question about how to accomplish the following. Is it possible to get the value of // pseudo function that grabs the dir value
const installDir = getValue(['install', 'dir'])
// if app is already installed, use existing config as defaults
const defaults = require(installDir + '/config.js') || getDefaults()
program
.command('install <dir>')
.option('--secure', 'set to secure', defaults.secure)
.action((dir, options) => { ... }) |
I don't see an easy way to get a subcommand argument without parsing. A different approach might be to read the local configuration in a |
A couple more related approaches I saw recently:
Mentioned in: #1748 (comment)
|
Problem
Adding support of config files when you have an option with a default value and you want the following priorities, is complicated.
Because commander has no built-in support for config files, you can't use all its power, because options object that you get by calling
opts()
contains mixed default parameters (point 1 from above) and cli parameters (3) and you should manually inspect each option with default to correctly apply the config file (2) parameters. I would like if commander could do this for me.The current workaround looks like this:
but it is incomplete: if I specify
--option-with-default "default value"
in the command line the code above will think that it is default value and override it with the config option value, which is not expected. To solve this problem I'll have to parse options manually, which effectively makes commander useless in such scenario.Proposal
The proposed API could looks like:
The examples for the CLI definition below:
.config.js
does not existprog
Result:
prog --default a
Result:
prog --option b
Result:
.config.js
containsoption
prog
Result:
prog --default a
Result:
prog --option b
Result:
.config.js
containsdefault
prog
Result:
prog --default a
Result:
prog --option b
Result:
The text was updated successfully, but these errors were encountered: