npm publish fails / npm publish <folder> publishes from cwd folder

[unlight]

I have configured semantic-release/npm with pkgRoot = './dist'
On CI/CD command is npx -w my-package semantic-release

CI run 28 Logs:

[1:20:01 PM] [semantic-release] [@semantic-release/git] › ℹ  Prepared Git release: my-package-v1.1.2
[1:20:03 PM] [semantic-release] › ✔  Created tag my-package-v1.1.2
[1:20:03 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/npm"
[1:20:03 PM] [semantic-release] [@semantic-release/npm] › ℹ  Publishing version 1.1.2 to npm registry on dist-tag latest
+ my-package@0.0.0-dev.1
[1:20:04 PM] [semantic-release] [@semantic-release/npm] › ℹ  Published my-package@1.1.2 to dist-tag @latest on https://registry.npmjs.org/

Note: my-package@0.0.0-dev.1 is a result of npm publish

CI run 29 Logs:

2022-07-31T13:06:50.035Z semantic-release:plugins options for @semantic-release/npm/verifyConditions: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.040Z semantic-release:plugins options for @semantic-release/npm/prepare: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.040Z semantic-release:plugins options for @semantic-release/npm/publish: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.041Z semantic-release:plugins options for @semantic-release/npm/addChannel: {
  pkgRoot: '/root/packages/my-package/dist'
}
[1:06:51 PM] [semantic-release] [@semantic-release/npm] › ℹ  Write version 1.1.2 to package.json in /root/packages/my-package/dist
v1.1.2
[1:06:52 PM] [semantic-release] › ✔  Completed step "prepare" of plugin "@semantic-release/npm"
[1:06:52 PM] [semantic-release] [@semantic-release/git] › ℹ  Found 1 file(s) to commit
2022-07-31T13:06:52.112Z semantic-release:git commited files: [ 'CHANGELOG.md' ]
[1:06:52 PM] [semantic-release] [@semantic-release/git] › ℹ  Prepared Git release: my-package-v1.1.2
[1:06:53 PM] [semantic-release] › ✔  Created tag my-package-v1.1.2
[1:06:53 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/npm"
[1:06:53 PM] [semantic-release] [@semantic-release/npm] › ℹ  Publishing version 1.1.2 to npm registry on dist-tag latest
npm ERR! code E403
npm ERR! 403 403 Forbidden - PUT https://registry.npmjs.org/my-package - You cannot publish over the previously published versions: 0.0.0-dev.1.
  shortMessage: 'Command failed with exit code 1: npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/',
  command: 'npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/',
  escapedCommand: 'npm publish "/root/packages/my-package/dist" --userconfig "/tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc" --tag latest --registry "https://registry.npmjs.org/"',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: 'npm ERR! code E403\n' +
    'npm ERR! 403 403 Forbidden - PUT https://registry.npmjs.org/my-package - You cannot publish over the previously published versions: 0.0.0-dev.1.\n' +
    'npm ERR! 403 In most cases, you or one of your dependencies are requesting\n' +
    'npm ERR! 403 a package version that is forbidden by your security policy, or\n' +
    'npm ERR! 403 on a server you do not have access to.\n' +
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false,
  pluginName: '@semantic-release/npm'
}
Error: Command failed with exit code 1: npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/
##[error]Process completed with exit code 1.

As you can see from logs version 1.1.2 is correctly written to package.json in dist folder, but following command is failing

npm publish "/root/packages/my-package/dist" --userconfig "/tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc" --tag latest --registry "https://registry.npmjs.org/"

It's trying to publish 0.0.0-dev.1 which is in package.json in /root/packages/my-package (equals to process.cwd())

But if I execute this command locally, it works without errors.

Corresponding npm publish command:

const result = execa(
  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd, env, preferLocal: true}
);

Looks like that execa has issues with cwd or preferLocal (or its combination).
Update: Some additional research results here in https://github.com/semantic-release/npm/issues/504#issuecomment-1201540336

[NullVoxPopuli]

I just ran in to this as well.

my stack:

   at makeError (<repo>/node_modules/.pnpm/execa@5.1.1/node_modules/execa/lib/error.js:60:11)
    at handlePromise (<repo>/node_modules/.pnpm/execa@5.1.1/node_modules/execa/index.js:118:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async module.exports (<repo>/node_modules/.pnpm/@semantic-release+npm@8.0.3_semantic-release@18.0.1/node_modules/@semantic-release/npm/lib/prepare.js:21:3)}

This fails when running not through execa as well:

❯ npm version 6.1.2 --userconfig /tmp/28e1f8c78e58a4699a9d64a8774e2e50/.npmrc --no-git-tag-version --allow-same-version
ember-statechart-component
v6.1.2
npm ERR! Cannot read properties of null (reading 'matches')

execa is on version 6 something -- could it be relateD?

[travi]

execa is on version 6 something -- could it be relateD?

the range defined for this project's dependency on execa does not allow v6, so something is misconfigured in your project. you will need to resolve that first before further investigation could be done within the context of semantic-release

[unlight]

I made some research.
I patched node_modules/@semantic-release/npm/lib/publish.js

const command = `npm publish ${basePath} --userconfig ${npmrc} --tag ${distTag} --registry ${registry}`;
require('child_process').execSync(command);

Same result, it maybe issue with npm itself (my version is 8.11.0), also I do no see any related bug fixes in changelog of execa (after v5)

I've tried another trick:

// npm publish --userconfig /tmp/xxx/.npmrc --tag latest --registry http://localhost:4873/ # {cwd: /root/packages/my-package/dist}
const result = execa(  'npm',
   ['publish', '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}
 );

And it works!

So it definitly something wrong with npm and npm publish <folder> command, npm publish for current folder works ok.

[travi]

But if I execute this command locally, it works without errors.

in your local test, did you execute directly from your dist directory, or with the same command that semantic-release is running and from the effective cwd?

i'm wondering if what you are describing is either a breaking change in npm or a bug that should be reported to them.

my version is 8.11.0

the npm plugin for semantic-release defines a direct dependency on npm, so the version installed because of that would have more impact on this situation than your globally installed version. you can determine which version is locally installed with the command npm ls npm. if you are using our recommendation of executing with npx instead of installing locally to your project, that would mean that the latest version within the defined range will be used.

[unlight]

But if I execute this command locally, it works without errors.

in your local test, did you execute directly from your dist directory, or with the same command that semantic-release is running and from the effective cwd?

same command that semantic-release is running and from the effective cwd

my version is 8.11.0

Yes, sorry, it's my global version. npm ls npm shows npm@8.15.1

[unlight]

i'm wondering if what you are describing is either a breaking change in npm or a bug that should be reported to them.

In 1st comment I described behavior of underlying npm (npm@8.15.1)
In research global npm was used.
Same results.

[Badisi]

Just got the exact same issue yesterday and took me the whole day to figure it out.

This has nothing to do with execa and is more related to an npm issue while using workspaces.
You can see the detailed explanation and reproducible repo here : npm/cli#5745.

Nonetheless, this can be fixed in @semantic-release/npm by replacing cwd with cwd: basePath here: publish.js#L26.
As I don't see why the publish command should be executed from a different cwd anyway...

[Badisi]

@travi, would you accept a PR to fix that ? (ie. replace cwd with cwd: basePath)

[travi]

As I don't see why the publish command should be executed from a different cwd anyway...

are you considering the pkgRoot option in that statement?

This has nothing to do with execa and is more related to an npm issue while using workspaces.

@unlight @NullVoxPopuli are you using npm workspaces in the projects where you encountered this situation?

[NullVoxPopuli]

pnpm workspaces, but ya

[Badisi]

@travi, yes I'm considering it.

What I meant is:

Looking at the code here (publish.js#L26):

const basePath = pkgRoot ? path.resolve(cwd, pkgRoot) : cwd;
[...]
const result = execa(
  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd, env, preferLocal: true}
);

We can see that :

• basePath is the final dist location (nb: it is taking pkgRoot into account)
• execa is executed from cwd
• npm publish command is given a basePath to publish

But why would execa be executed from a different path than basePath ?

---

A fix to this issue (also suggested by @unlight) would be to make sure that execa is executed directly from the dist path.

Something like:

  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}

or simply:

  'npm',
  ['publish', '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}

[Badisi]

As it is a blocking issue for me, I have made a PR to fix it.

@travi, I would be very pleased if you could approve it. Thanks 🙏

[donaldxdonald]

any progress? 💦

[Badisi]

For anyone interested

As no action were taken here and it was blocking for me, I'm using this temporary fix :

package.json#L28

{
  "scripts": {
    "postinstall": "node patch-semantic-release-npm.js"
  }
}

patch-semantic-release-npm.js#L1-L24

/**
 *  TODO: this whole patch can be removed when https://github.com/semantic-release/npm/pull/531 is fixed
 */

const { readFileSync, writeFileSync, existsSync } = require('fs-extra');
const { green, red, gray } = require('@colors/colors/safe');
const { dirname, join } = require('path');

const filePath = join(dirname(require.resolve('@semantic-release/npm')), '/lib/publish.js');
if (existsSync(filePath)) {
  let data = readFileSync(filePath, { encoding: 'utf8' });

  if (!data.match(/cwd: basePath/gm)) {
    data = data.replace("'publish', basePath,", "'publish',");
    data = data.replace("cwd, env,", "cwd: basePath, env,");
    writeFileSync(filePath, data, { encoding: 'utf8' });
    console.log(green('success'), '@semantic-release/npm patched.');
  } else {
    console.log(green('success'), '@semantic-release/npm already patched.');
  }
} else {
  console.error(red('error'), 'cannot patch @semantic-release/npm.');
  console.error(gray(`"${filePath}" not found`));
}