Composer: circular dependencies and COMPOSER_ROOT_VERSION

[weirdan]

Package ecosystem
composer

Package manager version
irrelevant

Language version
irrelevant

Manifest location and content prior to update
/composer.json
https://github.com/weirdan/psalm/blob/051125971bce3db49469e0f2c6d4b2429dc3464f/composer.json

dependabot.yml content
https://github.com/weirdan/psalm/blob/051125971bce3db49469e0f2c6d4b2429dc3464f/.github/dependabot.yml

Updated dependency
irrelevant

What you expected to see, versus what you actually saw
Some dependencies updated (pull requests created), e.g. for symfony/process

Native package manager behavior
Locally it would update with no issues, as Composer guesses root package version from git branch (dev-master in this case).

Images of the diff or a link to the PR, issue or logs
No PRs created. Github-native Dependabot emits the following error:

It can also be reproduced locally in dependabot-core repo by running

$ mv .git .git.backup && bin/dry-run.rb composer weirdan/psalm; mv .git.backup .git

=> fetching dependency files
=> dumping fetched dependency files: ./dry-run/weirdan/psalm/
=> parsing dependency files
=> updating 27 dependencies: amphp/amp, amphp/byte-stream, composer/package-versions-deprecated, composer/semver, composer/xdebug-handler, dnoegel/php-xdg-base-dir, felixfbecker/advanced-json-rpc, felixfbecker/language-server-protocol, netresearch/jsonmapper, nikic/php-parser, openlss/lib-array2xml, sebastian/diff, symfony/console, webmozart/path-util, bamarni/composer-bin-plugin, brianium/paratest, php-parallel-lint/php-parallel-lint, phpdocumentor/reflection-docblock, phpmyadmin/sql-parser, phpspec/prophecy, phpunit/phpunit, psalm/plugin-phpunit, slevomat/coding-standard, squizlabs/php_codesniffer, symfony/process, weirdan/phpunit-appveyor-reporter, weirdan/prophecy-shim

=== amphp/amp ()
 => checking for updates 1/27
 => latest available version is 2.5.2
 => handled error whilst updating amphp/amp: dependency_file_not_resolvable {:message=>"Your requirements could not be resolved to an installable set of packages.\n  Problem 1\n    - vimeo/psalm 1.0.0 requires openlss/lib-array2xml ^0.0.10||^0.5.1 -> satisfiable by openlss/lib-array2xml[0.0.10, 0.5.1] but these conflict with your requirements or minimum-stability.\n    - Installation request for psalm/plugin-phpunit ^0.13 -> satisfiable by psalm/plugin-phpunit[0.13.0].\n    - psalm/plugin-phpunit 0.13.0 requires vimeo/psalm dev-master || dev-4.x || ^4.0 -> satisfiable by vimeo/psalm[4.0.0, 4.0.0-beta1, 4.0.0-beta3, 4.0.1, 4.1.0, 4.1.1, 4.2.0, 4.2.1, 4.3.0, 4.3.1, 4.3.2, 4.4.0, 4.4.1, 4.5.0, 4.5.1, 4.5.2, 4.6.0, 4.6.1, 4.6.2, 4.6.3, 4.6.4, 4.7.0, 4.7.1, 4.7.2, 4.7.3, dev-master, 4.x-dev].\n    - Can only install one of: vimeo/psalm[4.0.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.0.0-beta1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.0.0-beta3, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.0.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.1.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.1.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.2.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.2.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.3.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.3.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.3.2, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.4.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.4.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.5.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.5.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.5.2, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.6.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.6.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.6.2, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.6.3, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.6.4, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.7.0, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.7.1, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.7.2, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.7.3, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[dev-master, No version set (parsed as 1.0.0)].\n    - Can only install one of: vimeo/psalm[4.x-dev, No version set (parsed as 1.0.0)].\n    - Installation request for vimeo/psalm No version set (parsed as 1.0.0) -> satisfiable by vimeo/psalm[No version set (parsed as 1.0.0)]."}
............ and another 26 times, for all subsequent dependencies .........

If .git folder is not removed, Composer picks up current branch of dependabot-core repo and uses that to construct the version (the output above would show vimeo/psalm dev-main instead of vimeo/psalm 1.0.0).

Root cause
The problem is caused by the circular dependency between vimeo/psalm and psalm/plugin-phpunit (see its manifest: https://github.com/psalm/psalm-plugin-phpunit/blob/1ce1ef2c3fe8bed6ddaba1607c00b48a52145023/composer.json), where the latter requires vimeo/psalm:dev-master, but Composer fails to detect current version and falls back to 1.0.0 which is incompatible with psalm/plugin-phpunit.

Possible fix
This can be solved by providing root package version via COMPOSER_ROOT_VERSION environment variable (see https://getcomposer.org/doc/03-cli.md#composer-root-version), setting it to dev- + the target branch.

[weirdan]

Hardcoding the version to dev-master allows dry-run to complete successfully with expected results.

diff --git a/composer/lib/dependabot/composer/update_checker/version_resolver.rb b/composer/lib/dependabot/composer/update_checker/version_resolver.rb
index ac02f8f5a..e6f7e5c97 100644
--- a/composer/lib/dependabot/composer/update_checker/version_resolver.rb
+++ b/composer/lib/dependabot/composer/update_checker/version_resolver.rb
@@ -137,7 +137,10 @@ module Dependabot
                 dependency.name.downcase,
                 git_credentials,
                 registry_credentials
-              ]
+              ],
+              env: {
+                "COMPOSER_ROOT_VERSION" => "dev-master"
+              }
             )
           end
         end

[weirdan]

This is basically a duplicate of #2008, but I feel all the data provided here + suggested solution merits keeping this one open.

[weirdan]

It's been a while since this was opened. Is it on the roadmap?

[weirdan]

It looks like there was an undocumented feature some time ago, that allowed you to set it up like this:

registries:
  composer-root-version:
    type: php-environment-variable
    env-key: COMPOSER_ROOT_VERSION
    env-value: dev-master

dependabot-core/composer/lib/dependabot/composer/file_updater/lockfile_updater.rb

Lines 455 to 459 in dba839c

	def credentials_env
	credentials.
	select { |c| c.fetch("type") == "php_environment_variable" }.
	to_h { |cred| [cred["env-key"], cred.fetch("env-value", "-")] }
	end

But now dependabot says it's not valid according to the config file schema.

[weirdan]

An alternative approach (that does not require any environment variables) is to switch from the shallow clone that confuses Composer to a blobless (--filter=blob:none) clone¹. Renovate uses it, and it doesn't suffer from this problem.

Footnotes

1. https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ ↩