Audit and/or validate certain "unique" details of package.json

[paceaux]

I had this issue come up today, regarding gulp-jest.

There is a particular bug with gulp-jest, and a pull request was submitted. Unfortunately, that PR has been sitting there for quite some time.

So two folks took it upon themselves to make a package from that PR: gulp-jest-acierto, and @jacobq/gulp-jest

When I search on NPM for "gulp jest", all three packages show up, and it's somewhat obvious which one is the "real" gulp-jest because of the number of downloads.

However:

1. All three packages have the exact same readme, which means two of them have incorrect installation instructions
2. Two of the packages (@jacobq/gulp-jest , gulp-jest) have the same name in their package.json
3. All three packages show things that are identical which one would expect to be unique:

• Name
• Author
• website
• Repository URL

I think this reveals a few problems with the NPM registry and the package.json

1. There's no validation that the name field matches the published package name
2. There's no validation that multiple packages have identical names in the package.json
3. There's no validation that the publisher is the same as the author
4. There's no audit to confirm if multiple packages are using the same repo url.

So I think there's some solutions to discuss

1. Ability to flag NPM packages as having a relationship, such as a fork, to another
2. Ability to flag, from the community, that a package may not be maintained any more
3. Auditing of package.json against claimed repos and raising warnings in NPM
4. Deprecate packages not maintained more than a year, and require the package owner to "re-up" ownership, or allow another member of the community to take ownership.
5. connect NPM accounts to github accounts so that author == publisher.

[ljharb]

If it already doesn't do this, imo npm publish should refuse to publish by default when the package.json's "name" doesn't match the package name being published.

[ethomson]

More deeply, the name field is used to determine the... well, name. The name field is unique across these three packages and matches the name that they're published as.

[paceaux]

Maybe I need some education, then.

for the "scoped" gulp-jest, the name is still gulp-jest. Is this expected for scoped packages?

[MylesBorins]

@paceaux TBH I'm not 100% what is going on but in the published package the name is

"name": "@jacobq/gulp-jest"

As per our docs the scope needs to be included in the name when being published

https://docs.npmjs.com/creating-and-publishing-an-organization-scoped-package

To verify the package is using your organization scope, in a text editor, open the package's package.json file and check that the name is @your_org_name/<pkg_name>, replacing your_org_name with the name of your organization.

So I'm unsure as to why the source of this project doesn't have that in place and is publishing, that might be worth opening an issue on the repo to enquire about how they are publishing

[paceaux]

@MylesBorins The fact that I have no easy way to trace the @jacobq/gulp-jest package to its source code surely speaks to the fact that there's some sort of problem here.

BTW, for gulp-jest-acierto I submitted a PR to update the README and package.json, and I've reached out to the maintainer of @jacobq/gulp-jest and also asked him to update the package.json and readme .

[MylesBorins]

I think the challenge is that the source on GitHub doesn't match what was published, and as things exist right now there isn't really a way for us to validate that. What we can validate is what is published, when it is published.

It seems to me, and I could be mistaken, that a lot of the issues you have run in to here are issues with the maintainers of the packages (e.g. not pushing published code to GitHub). We have discussed some possibilities of verifying that a package is derived from a specific commit on GitHub, perhaps that would solve some of the issues you have run into.

[ethomson]

I think this reveals a few problems with the NPM registry and the package.json

Yes, you're right. This is a disappointing experience, I agree. Just to clarify the rationale here - I'm sure that many of you who are reading this already know these, but it bears mentioning for completeness in case somebody reading this thread is not familiar with this:

There's no validation that the name field matches the published package name
There's no validation that multiple packages have identical names in the package.json

The published package names in your example are all unique, and correspond to their package names:

https://www.npmjs.com/package/gulp-jest-acierto has a name field of gulp-jest-acierto, and
https://www.npmjs.com/package/@jacobq/gulp-jest has a name field of @jacobq/gulp-jest

There's no validation that the publisher is the same as the author

No, that's correct. This is much harder to validate the correctness of. An "author" may be an organization. Or the package.json may specify the original author or founder of a project for historical or honorific reasons, while a new maintainer has taken over and is publishing the package. Or a package may have multiple maintainers.

There's no audit to confirm if multiple packages are using the same repo url.

No, that's correct. This is also very hard to use as a signal for correctness. Many people publish multiple packages out of a single repository (the "monorepo" pattern).

Thinking through some of the changes that you've proposed - which I think are possibly each individual pieces of feedback that should be evaluated separately:

Ability to flag, from the community, that a package may not be maintained any more

I'm not opposed to this, but it feels like this is only the first step of this functionality. I guess my bigger question is what actually happens when people have flagged this? ie, what action do you want npm to take here if a package is no longer maintained?

connect NPM accounts to github accounts so that author == publisher.

I definitely want to be able to have my accounts connected, but I don't see how these two things necessarily track yet. I don't think that an author and publisher are necessarily the same (intentionally) regardless of account linkage?

[paceaux]

No, that's correct. This is much harder to validate the correctness of. An "author" may be an organization. Or the package.json may specify the original author or founder of a project for historical or honorific reasons, while a new maintainer has taken over and is publishing the package. Or a package may have multiple maintainers.

Maybe, then, there needs to be ways to distinguish: creator, maintainer, publisher.
the gulp-jest and offspring demonstrate that there is no way to tell the difference.

No, that's correct. This is also very hard to use as a signal for correctness. Many people publish multiple packages out of a single repository (the "monorepo" pattern).

Maybe what I'm suggesting, then, is the ability to flag this pattern. Because in the case of the three gulp-jests, this is definitely not a monorepo pattern.

• validate if other packages use this repo url
• if yes, a "is monorepo" flag

ie, what action do you want npm to take here if a package is no longer maintained?

Have "phases of deprecation/ handoff"

1. flag it as, "appears no longer maintained" for XXXX time. Notify owner, "this is no longer maintained, do you want to continue owning it?". Give owner XXXX time to respond
2. Flag it as, "confirming current maintainer", Ask owner, "are you looking for a new maintainer", give owner XXXX time to respond.
3. Flag it as, "Seeking new maintainer". Allow NPM account holders to request to be a maintainer. give owner xxxx time to choose a maintainer from those who have requested. If no owner chosen, first-come, first-serve.

Maybe this is a bit contrived, but it amounts to:

• visibility that a package's relevance is in question
• opportunity for the owner to maintain their package
• opportunity for someone interested in the package to take ownership as benevolently as possible

. I don't think that an author and publisher are necessarily the same (intentionally) regardless of account linkage?

that's a fair point, which I think further suggests the need to distinguish creator, maintainer, publisher

[MylesBorins]

Some may not agree with me, but I do not believe it is the role of npm to manage deprecation and handoff. We have an existing documented process for handling disputes and have a separate discussion where we have pointed out the existing policy, but we are going to see if there are ways for us to improve this (this is on our triage agenda).

I think making it clearer how to handle these types of conflicts is good, but I don't think that in all cases it is appropriate for us to take action. In many cases the right move is to fork a rerelease, rather than take over an existing popular but unmaintained package.

[paceaux]

"Some may not agree with me, but I do not believe it is the role of npm to manage deprecation and handoff."

I can understand the desire to not take a hard position on managing deprecation and handoff; it's a responsibility and comes with overhead. It may not be something to think about right now, but I would ask that NPM keep in mind that software developers retire and also die (ideally not all at once). Handoff can be a more fickle thing if the maintainer isn't available; hence my suggestion and a drawn-out community-encouraged approach.

Setting aside the issue of deprecations and handoffs, I think I'd like to see us find a better way to address the scenario that I encountered:

• main package isn't being updated
• main package has an issue that requires a solution
• solution can't be merged into main package for var reasons;
• someone wants to publish that solution for others to see

It doesn't seem like NPM has clear guidance or indicators in a package.json to address how someone goes about correctly associating their package to the main one, identifying issues that the child package addresses, nor is it clear that NPM has a strategy for how the repository url is meant to be used (as I keep hearing folks talk about monolith repos, yet this was no such case)