Support custom register for composer

[alexander-akait]

What would you like Renovate to be able to do?

Will be great if renovate will support custom registers.

Example:

{
  "name": "wp-theme",
  "type": "wordpress-theme",
  "license": "MIT",
  "description": "Description.",
  "homepage": "https://insteria.com",
  "repositories": [
    {
      "type": "composer",
      "url": "https://wpackagist.org"
    }
  ],
  "extra": {
    "installer-paths": {
      "./plugins/{$name}/": [
        "type:wordpress-plugin"
      ],
    }
  },
  "require": {
    "php": "^7.1",
    "composer/installers": "~1.6.0",
    "wpackagist-plugin/breadcrumb-navxt": "~5.0.0"
  }
}

Describe the solution you'd like

I don't know.

Describe alternatives you've considered

I don't know.

Additional context

Doesn't exist.

[rarkins]

In the above example, I assume that this part:

  "repositories": [
    {
      "type": "composer",
      "url": "https://wpackagist.org"
    }
  ],

translates to: "use https://wpackagist.org for all package lookups"?

But is it a complete mirror? It looks like it isn't?

• Is Renovate/composer somehow meant to guess which packages are from wpackagist vs packagist?
• Or, is Renovate/composer meant to try wpackagist first and for all the ones that fail it then tries the regular packagist?

[alexander-akait]

@rarkins Hm, i think we need look on composer logic. I can do this in near future.

[rarkins]

@Jamesking56 could you add your use case here? You mentioned using a Satis repository but I'm not sure if you use it selectively or for all packages.

I think this issue is a prerequisite before we can implement authentication for private packages in #2354.

To implement this, once we have settled on the correct logic to use:

1. Extend the composer extract function to include necessary host information, probably in the purl field

2. Extend the packagist lookup function to recognise the extracted host information and do the right thing.

Links on the composer.json repositories field:

• https://getcomposer.org/doc/04-schema.md#repositories
• https://getcomposer.org/doc/05-repositories.md

This text seems to answer my question from above:

Note: Order is significant here. When looking for a package, Composer will look from the first to the last repository, and pick the first match. By default Packagist is added last which means that custom repositories can override packages from it.

It seems like this is a try/fail/try type of approach, i.e. it's expected that you try the first in repositories even if it has only 1 or 2 of the packages, and eventually you fall back to using Packagist as the default. None of our other lookups so far use this approach so I need to think if we need to make any internal API changes to accommodate it, or embed all the logic into lib/datasource/packagist.

The second challenge is that the list of reposotories can potentially be quite long and I'm not sure if appropriate to embed it into the purl.

I had expected to support only type "composer" to start with, i.e. not type vcs or package or pear. But I wonder if Satis is completely compatible with our current Packagist source code? More on Satis here

[rarkins]

I've started this moving in #2650 and discovered a few things.

First, packagist.org seems to include a "convenience" url for packages (e.g. https://packagist.org/packages/foo.json) that other ones may not, so we need to implement "proper" lookups.

I haven't found any foolproof guide to this so far, but hopefully this is right:

• Retrieve packages.json, e.g. https://wpackagist.org/packages.json
• Fetch each of the provider-includes in order provided, e.g. https://wpackagist.org/p/providers-2018-09$de6aa4f251f3197cce9accffae1882053fc664cb352a94f90e9c7b23cfda336e.json
• Find the key for the package you need and then look it up by hash: https://wpackagist.org/p/wpackagist-plugin/1337-rss-feed-made-for-sharing$e9b6c98c63f99e59440863a044cc80dd9cddbf5c426b05003dba98983b5757de.json

Because these URLs include hashes, they can be cached indefinitely to reduce the load.

[renovate-bot]

🎉 This issue has been resolved in version 13.101.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀