Rediscuss if error message changes are semver-major

[Fishrock123]

See #3374

Opening here to tag for the ctc agenda.

[Fishrock123]

Fwiw I haven't landed 20285ad in v5.x for this (v5.1.0) release because I'm a little less sure about it.

[cjihrig]

I think #3374 was a broad enough change to constitute a semver-major. We don't generally change error messages just for the sake of changing them. I also think that if an error message changes as the side effect of a bug fix, that shouldn't constitute a semver-major.

[silverwind]

It's likely to have code like this in the wild:

if (err && /something/.test(err.message))
  // do something specific to that error

In this regard, one could see the message as part of the API. So it should be tagged as major or reverted, but generally I'm in support of consistency.

[silverwind]

I tagged #3374 and #3727 as semver-major.

[silverwind]

cc @micnic

[OrangeDog]

Checking error strings is bad practice, and I don't see that we should necessarily support it.
As well as minor changes, typos, improved explanations, error strings may also be localised.

If specific errors need specific handling, then error codes should be used instead.

[micnic]

I also agree that checking error messages is a bad practice, and I guess that in the wild it is somewhere used, so I think we should put it as a major change to minimize the frustration of those who check them when using a new version of node.

Maybe in the future we could add some prefixed error messages with a fixed code like this: [code: 0123] <Variable error message>

[OrangeDog]

Most errors already have a status code, an errno, an exit status, etc.

[Fishrock123]

TSC CTC punted it a week or so. Not going to be in the v5.1.0 release though.

[Fishrock123]

Related: #4311

[jasnell]

Believe this can be closed now.