feat: add parsed meta-data to returned properties

[shadowspawn]

Problem

We don't wish to offer extensive configuration knobs and dials for modifying parseArgs behaviour, but we don't currently return rich information for authors to implement features themselves.

See #84.

Solution

Return a property which contains an array of the parsed tokens detected in the args. This allows the author to inspect the meta-data to add additional checks to throw for repeated options (say), or reprocess the tokens entirely to add auto-arrays for repeated options (say).

The internal implementation uses the tokens to do the error checking and generate the values and positionals. "Eating our own dog food."

The extra details are opt-in and not returned by default.

Closes: #84

This includes some minor lint in the existing (upstream) documentation, including one which was independently fixed:

• doc: fix typo in util.parseArgs usage example nodejs/node#43332

Example

const { parseArgs } = require('@pkgjs/parseargs');
console.log(parseArgs(
    { tokens: true, strict: false, 
      options: { port: { short: 'p', type: 'string'} } }
));

% node index.js -p80 --foo run
{
  values: [Object: null prototype] { port: '80', foo: true },
  positionals: [ 'run' ],
  tokens: [
    {
      kind: 'option',
      name: 'port',
      rawName: '-p',
      index: 0,
      value: '80',
      inlineValue: true
    },
    {
      kind: 'option',
      name: 'foo',
      rawName: '--foo',
      index: 1,
      value: undefined,
      inlineValue: undefined
    },
    { kind: 'positional', index: 2, value: 'run' }
  ]
}

Examples Folder Experimental Only

I am not planning to commit the examples other than tokens.js in this PR. I'll submit them in separate PRs. I'll leave them in the PR until last minute as examples of using the tokens for custom processing.

[Eomm]

To improve the readability I would explain that we mean the kind=option token

Suggested change

	* option tokens
	* 'option' tokens

[ljharb]

rather than quotes, which i find confusing, how about linking the term to the appropriate section?

[Eomm]

Yeah, that works too.

I use the same syntax in the kind description to be homogeneous

[shadowspawn]

The compact POJO description is a bit subtle to read. How about an expanded version with all the properties listed?

Proposed expanded

A returned token has two properties which are always defined,
and some other properties which vary depending on the kind:

• kind {string} One of 'option', 'positional', or 'option-terminator'.
• index {number} Index of element in args containing token. So the
  source argument for a token is args[token.index].

An option token has additional parse details
for an option detected in the input args:

• kind = 'option'
• index {number} Index of element in args containing token.
• name {string} Long name of option.
• rawName {string} How option used in args, like -f of --foo.
• value {string | undefined} Option value specified in args.
  Undefined for boolean options.
• inlineValue {boolean | undefined} Whether option value specified inline,
  like --foo=bar.

A positional token has just one additional property with the positional value:

• kind = 'positional'
• index {number} Index of element in args containing token.
• value {string} The value of the positional argument in args (i.e. args[index]).

An option-terminator token has only the base properties:

• kind = 'option-terminator'
• index {number} Index of element in args containing token.

Old compact

The returned tokens have properties describing:

• all tokens
  • kind {string} One of 'option', 'positional', or 'option-terminator'.
  • index {number} Index of element in args containing token. So the
    source argument for a token is args[token.index].
• option tokens
  • name {string} Long name of option.
  • rawName {string} How option used in args, like -f of --foo.
  • value {string | undefined} Option value specified in args.
    Undefined for boolean options.
  • inlineValue {boolean | undefined} Whether option value specified inline,
    like --foo=bar.
• positional tokens
  • value {string} The value of the positional argument in args (i.e. args[index]).
• option-terminator token

(Also asked in: nodejs/node#43459 (comment))

[Eomm]

Suggested change

	* positional tokens
	* 'positional' tokens

[Eomm]

Suggested change

	* option-terminator token
	* 'option-terminator' token

[bcoe]

👍 assuming this has been synced with the upstream Node.js PR.

Congrats on the contribution to Node 🥳

[bcoe]

@shadowspawn may I merge this?

[shadowspawn]

Yes, it is up to date with upstream. (I'll get to it this weekend otherwise.)