One-off/local plugins

[surma]

🙋 feature request

While I haven’t written any meaningful plugins yet (mostly due to #4320), I feel like I will make use of them once I understand them. @jakearchibald and I have grown very fond of writing project-specific, one-off plugins when using Rollup (e.g. in PROXX, and a talk about one-off plugins), and I think approach would be similarly great for Parcel!

However, you need to currently install plugins as proper npm dependencies. Which means they either have to be published, or you put them in a sub-folder with their own package.json and then do something like npm i -S ./plugins/parcel-transformer-my-transformer.

💁 Possible Solution

My idea is to distinguish local/one-off plugins from proper npm plugins in .parcelrc by a leading ./:

{
  "extends": "@parcel/config-default",
  "transformers": {
    "*.png": ["./plugins/my-transformer.js"]
  }
}

[DeMoorJasper]

This is by design, we don't really want to allow this as it would be very hard to cache this and therefore would result in slower builds and cache invalidation that should not have happened. (there's a similar issue with locally linked packages and monorepo's which we haven't figured out yet).

I'd suggest using a monorepo (yarn workspaces or something similar) for this use-case. (this wouldn't solve the caching issue but is at least valid in the parcel config)

[mischnic]

#3397 (comment)

But having it in a monorepo/linked vs locally makes no difference regarding caching....

[surma]

I don’t quite agree with the reasoning, but it sounds like you have put a lot of thought into this and I can respect that :) And, after all, if necessary I can work with the local-install workaround.

[devongovett]

I'll reopen this for further discussion since we will have the same issues with caching for locally linked plugins in a monorepo anyway (which we're already working on).

I think the other reason we had for this was to encourage people to share their plugins on npm (or a private registry in their company) rather than just keeping them to themselves. However, I could see some very project specific plugins where maybe this doesn't make sense (and you could use the private flag in a monorepo package anyway).

cc. @jamiebuilds. What do you think?

[jamiebuilds]

The primary driver for me was to encourage community contributions. There are thousands of Babel plugins that companies have written internally and back when I would go around talking to them I found that they had built a lot of the same exact plugins.

I think that just the act of putting something in a “package” does a lot to encourage people to consider open sourcing their code. I think everyone wants to, but aren’t sure where to even begin when it’s just a file in their repo.

I was thinking that doing this would help “bootstrap” the Parcel community, and give it a stronger sense of collaboration. The requirement was always sorta intended to disappear as a limitation at some point.

Maybe there’s a better middleground though, maybe it’s okay that it’s a local path as long as that path is still structured like a package and that package is marked as private.

And maybe I’m totally wrong about the effect this will have on the community. It’s really just a hypothesis based on my experience with Babel plugins and multi-package repos. But it’s totally possible that it would just intimidate people out of creating plugins altogether. It’s hard to say.

On the point about caching, it’s definitely more difficult than just operating against a version, but it seems like you’ve already got a lot of the infrastructure there for caching those kinds of deps (Thanks @padmaia). If you think there’s a performance problem with it though, I might encourage a warning to be output that you are using a local plugin and those can be slower.

[jakearchibald]

This is a tough one. I've definitely got a lot of 'local' plugins that I use in various Rollup projects. Currently I copy and paste them, because they work for that project, but they aren't good enough to be widely usable. My intention is to publish them once they are, but maybe I'll get lazy. Parcel's strategy prevents me being lazy.

On the other hand, it makes rapidly developing a plugin pretty painful, especially at the start of a project where there may be a lot of plugins, that are later rolled into one, or one plugin that's split into many etc etc, so there's a barrier there that might make me think, "nah I'll just use Rollup instead".

[speigg]

As a workaround for this issue (#4879), I had to create a custom resolver that fixes the ordering of mainFields. Annoying that I need to jump through these hoops for such a trivial change.

[jamiebuilds]

@speigg Ideally Parcel would get most of those things "right" for everyone and you'd never need to configure them. But when Parcel gets something "wrong", we then need to know:

• For how many people is this "wrong"?
• Is there a "right" solution that will work for everyone/more people?
• Can the community adapt to our "wrong" solution and make something good out of it?
• If we made a specific change, what would be the actual implications of that?

If we made this sort of thing a configuration option, most people will just find some Stack Overflow question and make the same change in every config without ever raising it with us.

When someone does start a conversation about it, it will exist solely in a GitHub issue with 300 comments almost all of which will lack nuance. People will have their opinion based on their need at a point in time and will defend it without actually testing it in the community and answering the above questions.

These "hoops" you have to jump through force two things:

1. For the people unwilling to jump through those hoops, they'll probably find other solutions that will over time "normalize" the ecosystem (more on this in a second)
2. For the people willing to jump through those hoops, they're starting an important exploration to see if the change they want to make is something that Parcel should adopt.

For 1, I want Parcel to get things like module resolution right 100% of the time. But that's impossible today because every ecosystem does things in different ways, and few people are working towards normalizing any of that. The biggest damage that Webpack/Babel made to the ecosystem was allowing people to easily configure things that should never have been configuration points. The result is that lots of tools have half-baked solutions that only work for them, which creates pressure on other tools to support their half-baked solutions, which causes the ecosystem to degrade more and more over time. Forcing people to jump through hoops inverts that relationship and forces the community to examine these integration points.

For 2, if someone does jump through all these hoops, that means they are invested in the problem. That they think this is the "right" way to solve the problem. And because we push them towards publishing their solution as a package, they will likely have other users that they will talk to in issues, they'll iterate on the problem, they'll identify the problems with their solution, and then at the end of the day we have download stats/github stars/etc to show us how broad of a problem this is. All of that together will allow Parcel to make a really informed opinion.

[jamiebuilds]

@jakearchibald

On the other hand, it makes rapidly developing a plugin pretty painful, especially at the start of a project where there may be a lot of plugins, that are later rolled into one, or one plugin that's split into many etc etc, so there's a barrier there that might make me think, "nah I'll just use Rollup instead".

I think this is a good point, Parcel should be addressing how difficult it is to develop a plugin. There are ways of solving this I think:

• Ship a scaffolding mechanism in Parcel to easily create a plugin package
• Allow local/linked plugins to be run when NODE_ENV=development or something along those lines
  • Alternative: Don't allow local/linked plugins when CI=true or something
  • Alternative: --dangerously-enable-local-plugin=./...
  • Alternative: Do something to people's apps that they would never want to ship to production

[speigg]

@jamiebuilds thanks for explaining the rationale, it does make a lot of sense. For what it’s worth, I didn’t have to publish my custom resolver, all I had to do was add it to my dependencies like this:

{
  "dependencies": {
    "parcel-resolver-esm": "file:packages/parcel-resolver-esm"
  }
}

[hsablonniere]

Hey folks,

Nothing smart to add to this discussion but I recently tried to create some plugins and I used @speigg technique with npm file:. I would rather be able to use a relative path inside .parcelrc.

I think I get the "Parcel should just work for everyone" but I share many of @jakearchibald arguments:

I've definitely got a lot of 'local' plugins that I use in various Rollup projects.

On the other hand, it makes rapidly developing a plugin pretty painful...

[RamIdeas]

Just to add another use-case to this, the reason I want to reference a local plugin is so that I can test the plugin does what is expected

I don't want to unit test the individual functions I pass to new Transformer({ ... }) because I don't want to test internals of Parcel

[BenjaminWaye]

Giving more input to the above. Working for a larger corporation, currently with limited time to make a POC for a switch from webpack to parcel. Not sure if there is an issue with my parcel setup but it seems like a local plugin is the logical next step. Not being able to make one is going to break the use case.

[mischnic]

It is possible if you make your project a Yarn monorepo: https://github.com/mischnic/parcel-resolver-root

[BenjaminWaye]

You mean using your project as a template? I guess that seems like the only way to move forward but it makes for very time consuming workflow for a POC.

[devongovett]

We now have a section in the docs showing a couple ways to do this: https://v2.parceljs.org/configuration/plugin-configuration/#local-plugins https://parceljs.org/features/plugins/#local-plugins

[fregante]

What concerns me about the lack of this feature is that how is one supposed to test transformers if they must live in node_modules? Only via npm link?

[mischnic]

You mean because you can't test them inside of the transformer package itself?

With a monorepo, you could have packages for your project, the transformer and the transformer tests.

(I've updated the link above)

[AndyOGo]

Thanks @jamiebuilds for explaining your reasoning, it list valid points on a broader view.

On the other side, it makes it really painful to:

• develop a new plugin
• write some custom, non-generic plugin intentionally (for rewiring, backwards-compatibility, treating symptoms, etc.)
• use the Parcel API directly to make some customisations

[sahithyandev]

"nah I'll just use Rollup instead".

Sounds like a good solution.

[SquirrelJimmy]

"nah I'll just use Rollup instead".

Sounds like a good solution.

+1，parcel is foolish not open custom plugin in repo

[devongovett]

To be clear, it is already possible to make plugins that aren't published to npm, and are local to your project. This is documented here: https://parceljs.org/features/plugins/#local-plugins. We might be able to make this a little easier, but it can already be done.

[pastelsky]

Now that parcel isn't new and there are public plugins available, would you consider allowing specifying plugins through Parcel's node API? In most other bundlers (Webpack, Vite, Rollup), writing a plugin is essentially as simple as plugging in a function —

plugins: [function myPlugin(){...}]

In parcel, even with local plugins, the hassle of creating an npm package, local linking it etc. is still prohibitive for one-off plugins, or as a quick spike to developing new plugins that you want to open source.

In the process of encouraging plugins to be published to npm, I fear parcel might be making plugin development itself too cumbersome which means a lesser number of folks even care to try. Both webpack and rollup communities have done just fine here without a strong opinionated stance.

[devongovett]

It is not really possible to do through the node API, because functions cannot be serialized to send between worker threads. That's why plugins are npm packages (or at the very least, separate files), which can be loaded by each worker.

I think we probably should support loading relative paths for plugins though, basically as a shortcut for setting up a link: dependency or monorepo workspaces. This shouldn't be too hard, mainly relaxing some requirements in this file. If someone wants to send a PR that would be very helpful.