fix(util): Replace micromatch with picomatch to fix issues with negated globs

[danez]

Summary

This starts fixing some of the issues mentioned in #9464.

Problem 1:

[
  "**/*.js",
  "!(tests|coverage)/**/*.js
]

This snippet does not ignore anything right now when run through globsToMatcher because the second glob is not considered negated by micormatch/picomatch. Instead it is considered negatedExtglob. Unfortunatelly the scan() API of micromatch does not expose this detail. There is a PR for adding it (micromatch/picomatch#79).

The reason why I completely exchanged micromatch with picomatch is that picomatch has the option to return the parse-state, which makes scan completely unnecessary and shaving off a little bit of work. To visualize it:

// right now
scan(); // simple pre-scan to figure out what features the glob uses
matcher(); // simply forwards to picomatch(glob, {}, returnState: false)

// with this PR
picomatch(glob, {}, returnState: true);

So basically right now scan() scans for features in the glob and then parse has to do the same thing again for parsing. With the PR the state from parsing is used to figure out if the glob is negated.

So to conclude:
There are two options:

• either adding negatedExtglob to the scan API of micromatch and use this then similar to how it is used in this PR.
• use picomatch directly which supports negatedExtglob and also makes scan obsolete.

Not sure what you prefer?

Also micromatch was updated to 4.0.4 in devDepedencies because the negated extglob problem was fixed in this version in micromatch() (micromatch/picomatch#78, micromatch/micromatch#213).

Test plan

Added unit tests.

Missing

• Proper types for picomatch (fix(picomatch): Better types when returnState equals true DefinitelyTyped/DefinitelyTyped#52329)

[SimenB]

this is awesome, thanks for doing this!

I'm happy to change to using picomatch directly 👍

[SimenB]

I don't know if you dug into the git history here, but the place you're changing now comes from #10131. If there is some better way than sorta replicating what micromatch does internally than what we landed on there, that'd be awesome 🙂

[danez]

I don't know if you dug into the git history here, but the place you're changing now comes from #10131. If there is some better way than sorta replicating what micromatch does internally than what we landed on there, that'd be awesome 🙂

I only saw the inline comments but haven't really investigated. From the top of my head, I don't really know if this can be done with micro/picomatch directly. I will keep that in mind and maybe there is something simpler.

Edit: Also the reason for #10131 probably was updating from mm 3 to 4 because 3 had an internal in-memory cache so it would never parse the same glob twice. mm 4 on the other hand has no internal cache and keeps parsing over and over again.

So thinking more about it #10131 might actually have been a partial fix for #9457, but there are still direct calls to micromatch(), so maybe they need to be checked if called multiple times.

[SimenB]

Wonderful, thanks!

Edit: Also the reason for #10131 probably was updating from mm 3 to 4 because 3 had an internal in-memory cache so it would never parse the same glob twice. mm 4 on the other hand has no internal cache and keeps parsing over and over again.

Any chance of re-adding a cache? I guess we could also create some wrapper module around micromatch with a cache (sorta ish what #10131 did, but broader)

[SimenB]

do we still need this one?

[SimenB]

Published in 27.0.0-next.8. Thanks again @danez!

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.