Move to flat config

[fregante]

+

---

https://eslint.org/blog/2022/08/new-config-system-part-2/

The change has been out for a while and it might be a good time to start exporting it. I believe this will solve a lot of config-related issues:

• Error: Failed to load plugin '*' declared in 'BaseConfig... #546
• Failed to load config "xo/esnext" to extend from #404
• v0.13.0 crashes on plugin shared config #88
• Cannot disable eslint-plugin-node and eslint-plugin-n rules #613
• Unable to customize certain rules via shareable config #71
• Better solution for configs & plugins in dependencies #646

Some notes around this in #428 (comment)

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[chrisspiegl]

This seems to tackle a lot of things I have recently run into. Looking forward to the integration.

[spence-s]

I wanted to start this effort. But I think first, we need to think about what it means for xo as a whole. Ultimately, the original goal for xo (a shareable config that includes other configs and plugins I believe) is now complete. Eslint has also simplified their configuration a lot with the new flat config, and most of the original goals of xo have been solved with eslint.

What does this mean for xo then?

This means that we could potentially make xo nothing more than an eslint config and drop our cli/editor extensions and still hit 90% of xo use cases. Sherriff basically takes this approach here.

However, there are still a few features which justify continuing to maintain the xo cli. They are mainly the output formatter, the caching logic, and maybe a few other cases, I haven't discovered. I think this good enough reasons to keep maintaining a cli and editor plugin ecosystem.

Proposal:

Rewrite xo to:

1. export a single function which accepts an xo flat config (see 3 below) and returns an eslint flat config that contains most of the logic in xo now.
2. js and ts can be lint in a single linting pass whether it is ran with xo directly or with eslint the results will be the same. Some xo config options which don't affect lint results will be ignored when using the config function directly.
   • EDIT: May be some complications with our auto tsconfig generation for type aware ts linting which may make it difficult or impossible to achieve the same results with eslint and direct xo runs for ts, but I hope in the re-write we can smooth this behavior because its both a big source of complexity in xo and a source of a lot of bugs and confusion for xo users, but its a huge convenience tool.
3. the cli only needs to be simplified to the following.
   • Accept a single xo.config.js as a configuration mimicking the new eslint practice.
   • The xo.config.js file will export an array of xo configs
   • xo configs can largely stay as they are now and the changes will reflect the eslint config changes
     • overrides will be removed
     • extends will be removes
     • plugins will take an object now
     • There are probably other adjustments that will be found that need to be made to the configs/cli but they should be relatively minor.

Why a full rewrite?

Maybe not full, I think we can mostly keep the same cli.js file. xo, as is, is designed to run eslint multiple times for each xo configuration file group. However, this can be natively accomplished with eslint now with a single pass with much more efficiency. It will be much cleaner to rewrite the logic in our index.js and options-manager.js files from the ground up than to modify them to support the new stuff.

Backwards compatibility

We have a lot of options to maintain backwards compat behind a flag or option for a while if we want. Or we can simply rip the bandaid off in a version bump.

TL;DR: xo should start accepting its own "flat config" and be re-written for better interop and efficiency with eslint while keeping most (or all) of its current functionality and at the same time, be greatly simplified internally.

[fregante]

Love the idea, some of that is also covered/discussed in:

• Use and extend eslint's config files instead of mapping it to XO's own schema #428 (comment)

Such a change would also get the funding from:

• Use ESLint-style config options #319

[spence-s]

I've got a good start on this, I am going on vacation starting saturday for a couple of weeks, but I think I can get a draft PR for this before then, if not it should be in shortly when I get back.

Some more thoughts we can discuss and how my initial implementation looks:

1. I am going to put the flat config stuff behind a --flat flag (for the cli) and not change anything about normal xo usage, it will be totally opt in. The API will stay exactly the same (for editor integrations) but and editor integrations can expose the flat config on their own.
2. flat config will only read from xo.config.js and will use the standard cli options we have now as much as possible.
3. We can definitely export a function which creates an eslint config for easy interop for those who want to use eslint directly.
4. As implied by above we can lint js or ts in 1 pass and lint exactly the same. It appears to be more performant at first glance, but that needs a lot closer inspection, may be memory tradeoffs and other things.

Some potential downsides/pitfalls:

1. It's really too early to lean on flat config entirely for xo behavior. The API is still changing, and there are still bugs in the new system on ESLints side. So maintaining xo and xo --flat will require modifying code in 2 places when a change is made. So this is a bit of a maintenance burden. However, I think the tradeoff is worth it for the code simplification we'll eventually get as we move completely over to the flat implementation. I think it's worth bringing up to hear opinions on this tho. We could extract some common functionality better to make this a moot point.
2. Since we are taking a new direction with how xo behaves internally for the flat config, most of our tests can not be applied to it, so new tests will need to be written from scratch. This kind of sucks since we have a great test suite. But since we are doing a lot less work with the new code, maybe it won't be too painful to migrate a lot over.

New possibilities/features for xo:

1. As I have been thinking about ways to simplify xo and keep it modern and usable, I have been thinking about our handling of tsconfigs for type aware linting on any file. This is a cool feature of xo where you don't need one at all to get linting, we do all the work and generate a temp one for you if we can't find yours. However, it's a lot of lookup work on our end to find and create this file and has been a big source of bugs for xo.

   I was thinking with the flat config, we would change this behavior slightly. I would propose that we separate the type aware rules from the non-type aware rules, and only turn on the type aware rules if a user supplies a tsconfig. We could also expose a tsconfig option for convenience as its more clear than "parserOptions.project" field.

   If a user passes this field, then we turn on the type aware rules, if they don't then we turn off the type aware rules.

   Users still get linting on any ts file with no configuration, and we can avoid expensive and bug prone tsconfig lookups on every run.

   An added tradeoff of this approach is that users can now have the option to not use type aware linting at all (which they currently do not really have wo manually figuring out which rules to turn off). This can be a huge performance boost for linting large ts projects. The downside is the added configuration and not being type aware by default.

2. One thing that is tough about working with xo right now as a maintainer is we have one options object that gets passed around and changed by a bunch of different functions and its mentally really hard to keep up with. I am adding ts via jsdoc to the new flat internal code. Some of this typings gets passed to the legacy stuff as well and will help future maintainers (myself 😂 ) be more confident in their code. I think it would be good to make ts checks as part of ci validation as well.

[sindresorhus]

I was thinking with the flat config, we would change this behavior slightly. I would propose that we separate the type aware rules from the non-type aware rules, and only turn on the type aware rules if a user supplies a tsconfig. We could also expose a tsconfig option for convenience as its more clear than "parserOptions.project" field.

The point of TypeScript is to have types. Same applies to linting. I'm not very interested in supporting use-cases for not using the type aware rules.

Users still get linting on any ts file with no configuration, and we can avoid expensive and bug prone tsconfig lookups on every run.

ESLint may support types built-in at some point: eslint/eslint#16557

One thing that is tough about working with xo right now as a maintainer is we have one options object that gets passed around and changed by a bunch of different functions and its mentally really hard to keep up with. I am adding ts via jsdoc to the new flat internal code. Some of this typings gets passed to the legacy stuff as well and will help future maintainers (myself 😂 ) be more confident in their code. I think it would be good to make ts checks as part of ci validation as well.

I would be fine with using TypeScript for XO. That would be preferable over lots of JSDoc comments.

[yangmingshan]

I would love to see xo become to a simple eslint config, at least provide one.

I've been using xo as a eslint config set for years actually 😂

[spence-s]

Just an update here... I have worked on this a bit and have a version which works ok. It doesn't quite cover all xo options correctly yet, but it does cache ESLint instance and lint in 1 pass and has similar lint results (not quite 1 to 1).

Unfortunately the initial results show that a large and complex ESLint flat config is quite a bit slower than what xo does now, even though we can cache the eslint instance, which is surprising.

this implies that either:

1. I implemented things in a way that is very unoptimized for ESLint and it can be drastically improved by re-doing the work I have already done.
2. ESLint Flat Config internal parsing is just much slower.

It is likely a combination of both factors.

Since I discovered that its a big downgrade in performance I haven't worked on it anymore, and I've lost my motivation for this effort...

I think what I did was a good experiment, but now I think we maybe should take the path of least resistance with upgrade to flat config and use the current code base and just hook in the eslint compatibility libs and continue to use xo as it is. --- Not sure how the performance will look then.

https://github.com/spence-s/flat-xo for anyone who would like to review or try out the code.

[sindresorhus]

This is put on a break until ESLint improves flat config performance: #707 (comment)

[fregante]

Here are some other projects tackling this issue, for reference:

• feat!: use flat config antfu/eslint-config#250
• standard/eslint-config-standard@3d1284c

[sindresorhus]

This issue is now funded.

[spence-s]

Took a look at this repo again over the weekend. https://github.com/spence-s/flat-xo
The performance has definitely improved and things definitely seem to be more stable than the last time I looked (almost a year ago) and I feel like its ok to keep moving forward with this effort for me...

However, there are still some things to note.

It seems these plugins do not yet support flat configs. (see eslint/eslint#18093)

• eslint-plugin-comments
• eslint-plugin-import

Also, there is a ton of work to do for testing, many of xo's current tests are about parsing the old config type so those can't be ported over. Also the cli flags are changing so tests around those can't be ported either.

Anyway - I am going to continue work in the flat-xo repo, I welcome anyone who wants to contribute there, and when I get some better tests and stability, we can figure out the best way to get the code in the new repo under xo namespace/cli and all that good stuff.

If anyone wants to try to beat me to this I would encourage them to try, I think this thing is still a good while away from stability.

[voxpelli]

• eslint-plugin-import has a fork over at https://github.com/un-es/eslint-plugin-import-x that may update sooner: Support eslint flat config un-ts/eslint-plugin-import-x#29
• do you mean @eslint-community/eslint-plugin-eslint-comments @spence-s? I'm at the Community Core team so feel free to ping me if that's the case

[jlarmstrongiv]

Does this issue also cover the other config repositories?

• https://github.com/xojs/eslint-config-xo
• https://github.com/xojs/eslint-config-xo-typescript
• https://github.com/xojs/eslint-config-xo-react

[fregante]

Ideally yes, but I assume that support will follow later.

Other than eslint-config-xo, they only exist as standalone repos to be easily selectable in the old eslintrc-style config. The Flat Config is dynamic so one package can be configurable.

IMO eslint-config-xo can/should be converted first, and then this executable can follow, but I don't have full insight into what xo does.

[fisker]

Making the first move

xojs/eslint-config-xo#86
xojs/eslint-config-xo-space#10

[spence-s]

I will start putting together a draft PR that ports https://github.com/spence-s/flat-xo to this repo, which will enable xo itself to use a flat config.

Looks like it is mostly working as expected now with the exception eslint-plugin-import. There may be some other random rules that don't work quite right

There are a few other issues that need a discussion but those will be better had under the PR than this issue.

[voxpelli]

Since xo and standard has both been approaches that has avoided having to install eslint itself I think it would be great to collaborate on how to achieve that part for ESLint 9, considering that plugins etc are now direct dependencies rather than peer dependencies.

I summarized my thoughts here: neostandard/neostandard#16 (comment) (neostandard is the open governance fork of standard that we launched an early release of recently)

tldr is basically:

find an official solution together with the ESLint project, where we could easily bundle ESLint together with the config and expose it as our own non-configurable CLI, and have editors etc be able to pick that config up automatically (probably by eslint itself being able to pick up the configuration even when not launched through neostandard or xo)

EDIT: Made a standalone issue about this: neostandard/neostandard#33

---

Will review @fisker's PR:s and come with suggestions from what we did

[sindresorhus]

The bounty from #88 and #71 has been moved to this issue.

[voxpelli]

Opened an issue in ESLint about officially supporting xo / standard style packaging: eslint/eslint#18800

[fregante]

@sindresorhus @fisker How do you plan on integrating the TypeScript and React configs? I'm not sure that the direction taken in eslint-config-xo lends itself to multiple configurations (e.g. how would I combine xo-space, xo-react, xo-typescript?).

I'm currently a fan of https://github.com/antfu/eslint-config, which exposes a single global function which takes its own configuration. This is essentially the killer feature of xo-cli: abstracting the configuration of specific rules to a single property. For example xo's prettier: true is now possible via native eslint config.

If you take that approach, eslint-config-xo can have the entirety of the logic, configuration and plugins that xo has, and xo can just be a CLI adapter for it.

xo --prettier

would be strictly equivalent to

eslint .

// eslint.config.mjs
import xo from 'eslint-config-xo'

export default xo({
	prettier: true
});

so one can continue to choose to use xo as a tool or as a config, without the current drawbacks that this brings.

[voxpelli]

@fregante This was my feedback here xojs/eslint-config-xo#86 (review), we're taking that approach in neostandard as well:

export neostandard({
  semi: true,
  ts: true,
});

And currently having the CLI command generate such an output while discussing eslint/eslint#18800:

neostandard --semi --ts > eslint.config.js

Would love feedback to eslint/eslint#18800 from you as well 🙏

[fregante]

Would love feedback to eslint/eslint#18800 from you as well 🙏

I haven't because it wouldn't be particularly positive. After the flat config is implemented, there's very little reason to ship a CLI tool and maintain its entire ecosystem (IDE tools, etc).

Until now:

"scripts": {
	"test": "xo"
},
"devDependencies": {
	"xo": "^0.59.0"
}

After the flat config:

"scripts": {
	"test": "eslint . --config eslint-config-xo"
},
"devDependencies": {
	"eslint": "^9.0.0",
	"eslint-config-xo": "^0.59.0"
}

The only piece missing is having the --config flag resolve directly to a package in node_module. I'd lobby to support that instead.

What are the pros of a xo/neostandard CLI over this? I only see disadvantages.

[voxpelli]

The only piece missing is having the --config flag resolve directly to a package in node_module. I'd lobby to support that instead.

I agree on this, its one of the things I brought up as a possible solution in eslint/eslint#18800:

• neostandard, xo shared configs being possible to configure for ESLint in some other way, eg:
  • (ignoring the name of the arguments) eslint --shared-config=neostandard --shared-config-semi --fix
  • In package.json adding something like:

    "eslint": {
      "shared-config": "neostandard",
      "shared-config-options": {
        "semi": true
      }
    },

The tricky part about only eslint --config xo are two things:

1. Discoverability by tooling like VSCode (I ran into this quite a lot when configuring compatibility tests for neostandard against standard, as quite a few modules were specifying options when invoking standard rather than through discoverable options)
2. How to enable options like prettier: true to be specified

Would be great if you could add your 👍 to that approach @fregante in that issue, as current feedback from ESLint is that its not something that should be solved in ESLint, leaving us all having to add eslint.config.js files or rolling our own CLI:s

[fregante]

1. Discoverability by tooling like VSCode

Good point, then I think the best action would be:

// eslint.config.mjs
export {default} from 'eslint-config-xo'

And the usual:

"scripts": {
	"test": "eslint ."
},
"devDependencies": {
	"eslint": "^9.0.0",
	"eslint-config-xo": "^0.59.0"
}

Which is arguably longer than what XO/standard allows today, but also worth the ability to use ESLint-native tools, like IDE plugins and other stuff like eslint-interactive. The amount of work this would save you personally is probably worth dropping the current CLI idea.

A CLI tool would then just exist to set this up, as you suggested earlier: neostandard --semi --ts > eslint.config.js

2. How to enable options like prettier: true to be specified

If you need customizability, then a standard eslint.config.js already works, as shown just earlier.

[fregante]

What are the pros of a xo/neostandard CLI over this? I only see disadvantages.

To answer myself:

Pros

• Install one dependency
• Avoid eslint.config.js

Cons

• Install more IDE plugins
• No interoperability with existing tools (CLI, SaaS, …)
• Having to convince your boss/peers to "switch" to XO, making the two points above exponentially more painful

[spence-s]

I personally agree with the eslint team and would like to continue maintaining the CLI and the extension. It's just work I enjoy and has lots of advantages. In addition, auto resolution of configs and plugins was basically the very worst part about eslint and even xo suffered from it, I don't think bringing that back in any form is good.

XO cli basically originated from the limitations of the config and plugin auto-resolution being practically unshareable for multiple plugins. However, xo grew beyond that with a lot of different convenience things built in.

---

Note the the flat-xo implementation I have been poking at is like 90% complete and standard linting works great with both vscode and the cli and it even is "self linting" at this point https://github.com/spence-s/flat-xo

I will publish it as is under my scope (@spence-s/flat-xo) and write out some instructions for some beta testers who don't care about what's missing.

---

So what's the hold up?

1. prettier config does not yet support stylistic rules (this is the biggest issue for me as I always turn prettier on)
2. new ts eslint options still do not allow files that are NOT listed in tsconfig to use type aware rules. In fact the new way of doing things make it even more complicated to support this feature and get the performance boost from the tsconfig service.
3. webpack import resolvers - mainly because I never use it and don't want to test it, would be in favor of dropping this.

For config you can config for xo (using current xo cli and or vscode extension)

xo.config.js

export default [
    // customizations
    {  
        prettier: true, 
        semi: false.
        rules: {...}
    }
]

Note that this config you can do things like turn prettier on and off for a subset of files, but I do see possible benefits of making the xo exclusive rules global instead of a flat config for them.

for using eslint directly, you'll miss out on any ability to automatically get type-aware linting and have it read your node engine (not sure the latter is even useful anymore anyway).

But you can just

eslint.config.js

import {createEslintConfig} from 'xo'

export default createEslintConfig([
    // customizations
    {  
        prettier: true, 
        semi: false.
        rules: {...}
    }
])

and never use the CLI.

Also another big benefit of CLI is that we don't want to force current cli user to start installing eslint as a separate dep.

and finally, with the cli we can offer more config options such as allowing package.json to configure a subset of xo and not force users to create xo.config.js (not implemented in my package).

---

@fregante and @voxpelli you both bring up some great discussions and it's true that the xo cli in my flat implementation is only a very thin wrapper around eslint, but I really like building the vscode extension so I will continue to maintain that even if I am the only user of it and the cli allows room for some more complex config gathering for prettier and typescript etc... that may be a little out of scope for a plain eslint config.

[voxpelli]

@spence-s My hope was that we could find a solution for flat config based setups that could work for xo, (neo)standard and anyone else that wants to mimic that setup, without each of us having to build our own setup for CLI:s and editor extensions

[spence-s]

@spence-s My hope was that we could find a solution for flat config based setups that could work for xo, (neo)standard and anyone else that wants to mimic that setup, without each of us having to build our own setup for CLI:s and editor extensions

I hear you, and I agree to some extent, we've just already built those things, and I personally don't want to drop them 🤷🏻‍♂️

The xo vscode extension is pretty generic anyways. I've been debating pulling out the xo specific stuff and just making it a generic "eslint-lint-server" implementation that could use eslint or xo or any other tool that basically extends eslint lint results. It would also just be a standalone lint server that could work with any editor that supports LSP (most of them now).

I just really haven't had a compelling reason to take the time to do that. If you'd be interested in something like that for neostandard, I'd be happy to work on it, work with you, and/or help where I can.

And to the point being made by @nzakas on the eslint thread, the eslint vscode extension is actually maintained by microsoft, and not the eslint org, eslint has no control on that side of the things to implement the API you are asking for, you would need to approach that particular extension author for the implementation.

[voxpelli]

The standard org also has solutions already – extensions and standard-engine, but I agree with @fregante that I barely see the point with them now that flat configs solves the dependency bundling, so rather than porting them to flat configs and keep maintaining them I think we would be better off aligning on a common solution that solves what @wesleytodd brought up in the ESLint issue

[voxpelli]

things to implement the API you are asking for

I'm not asking for any specific API, I'm asking for collaboration on solving a pattern / use case

[spence-s]

I'm not asking for any specific API, I'm asking for collaboration on solving a pattern / use case

I just think we're at peak usability for users already and the pattern we are solving for is basically for the very few devs working on these tools right?

I mean with flat config and a custom extension and a custom cli:

The user gets :

1. options to use eslint tool chain
2. options to use xo tool chain

Proposed solution I feel you are asking for would, at minimum, require users to install 2 deps (eslint + the config). Then install the eslint editor integration. Then they'd configure some new config (for shared configs, that hasn't been settled yet) to tell eslint to default to the shared config with some kind of options.

That is the same amount of work they have to do if they want to use eslint toolchain anyway.

I am just not sure how this makes things better for users. I see the benefit to us as devs, just using the eslint built in stuff and avoid writing clis/integrations. However, since we have all that stuff in place already, I just don't feel that motivated to solve the problem.

Am I missing something?

[voxpelli]

I'm apparently not making myself understood, not sure if its a language barrier or something, maybe a native speaker like @wesleytodd describes it better here? eslint/eslint#18800 (comment)

[fregante]

@sindresorhus what's left here? It looks like the config is now flat:

• https://github.com/xojs/eslint-config-xo#usage
• https://github.com/xojs/eslint-config-xo-typescript

I think this is the last step:

• https://github.com/xojs/eslint-config-xo-react

[sindresorhus]

That and also doing the XO changes.

[voxpelli]

Closed eslint/eslint#18800 as I couldn’t get the point across and it just started to wear me down.

If anyone in xo is still interested in collaborating with (neo)standard on the next gen solution for this, then reach out in eg neostandard/neostandard#205 or somewhere. Until then I’ll unsubscribe from this thread 🙏