contesting allowing breakage in minor versions

[davidmarkclements]

If you special-case anything in an application it becomes communication overhead, and as soon as the chain of communication breaks (or someone forgets) it's not special-cased anymore and you're unknowingly open to unexpected breakage.

We're using mime-db in https://github.com/holepunchto/pear - an OSS P2P Runtime, a supply chain attack or really any kind of instability in the dependency tree can be existentially threatening because Pear updates are peer-to-peer, if it breaks it's broken on user machines (potentially forever). Therefore maintaining a policy decision that allows breaking changes in minor versions would mean we'd have to fork (which we'd rather not do).

Please reconsider.

For context:

• 🤖Automated update #328 (comment)
• fix: notice about semver and data sources #330
• https://x.com/wesleytodd/status/1814014140022956227

[wesleytodd]

Hey @jshttp/express-tc, just want to flag this to make sure we give this as much consideration as it is due.

I do believe that "data packages", or whatever you would call this where it ships a JS api and aggregated data from external sources, are not well supported via the controls we as package authors have. It means that we have a choice:

1. consider changes to existing mime types as majors but keep additions as minor
2. decouple the data from the JS api and version them independently (details TBD)
3. stay with the existing precedent and encourage folks to hard lock this dep in their package.json's

---

Specifically for the use case outlined above I think that there is a strong case for locking this dep in your package.json @davidmarkclements. No matter the decision we make, if changes can cause a ripple effect in the p2p network then I would say you will be better off locking deps, and I am not sure if that is isolated to this one, although I do see why this one is particularly difficult. ~1.53.0 would even be enough to ensure you don't get data updates unexpectedly for this package as we only ship data updates in minors.

[wesleytodd]

cc @davidmarkclements: jshttp/mime-types#126 (comment)

It is unfortunate that I did not realize it before releasing this, but we are working to cut support for older node versions which is requiring mass major version reving across our packages. This could have landed in that instead, which would have changed the medium term. Unfortunately that ship has sailed, and I think your use case was already broken. But going forward, I think we would totally be open to a new approach that decoupled the semver consideration from the data. If you are open to that I am sure we would love to see something that worked for all of us.

[ctcpip]

consider what is done for CLDR

unicode-org/rust-discuss#4
https://github.com/unicode-org/icu4x/blob/main/documents/design/data_pipeline.md

[ctcpip]

also relevant: https://github.com/tc39/proposal-canonical-tz#handling-time-zone-canonicalization-changes

[wesleytodd]

We decided in last weeks TC meeting to stick with the status quo for now. We have too much on our plates trying to get express@5 out the door to take this on right now. I think we all agree that this should be revisited in the future though, so I will keep it open for further discussion.

[broofa]

FWIW, both mime and mime-types have historically only done minor-version bumps when their mime-db dependency changes. I suspect this is mostly because that's all mime-db has ever done (minor releases).

[Aside: Currently mime doesn't even have a package.json-documented dependency on mime-db, because there was a two-year period where some useful changes to types weren't published. Instead, it just pulls mime-db.json directly from the HEAD of that repo. I'm not proud of that, but it solved a couple problems that I had as the mime maintainer.]

consider changes to existing mime types as majors but keep additions as minor

I actually like this idea, but implementing it gets a little tricky.

• mime-db historically has not made this distinction. Updates have only ever been minor.
• Changes that mime-db consider "minor" may result in "major" consequences in downstream packages. E.g. if a new mime type declares an extension that conflicts with those of an existing type. That may change the default type for that extension as seen through the lense of something like https://github.com/broofa/mime-score.

decouple the data from the JS api and version them independently (details TBD)

Why "TBD"? This is how mime|mime-type|mime-db work already. mime-db" is the data (the mime-db.json file), while mime and mime-types are the JS APIs.

[wesleytodd]

a two-year period where some useful changes to types weren't published

Our goal as the newly setup leaderhip group is to not let this happen. I would love if you would be willing to help us with that by being a collaborator on this (and any other related repos you have a stake in).

Why "TBD"?

Well we had a few relatively good options I think, but with our main focus being on releasing express@5 we have been trying not to spend too much of our limited time on these less impactful changes. There are trade offs with any change, so I just didn't wan tot say more than TBD to not state any opinion more strongly than I actually meant it.