Allow to run a satis build for one repo/package

[shochdoerfer]

Added an additional flag to generate the satis build for a single package. This helps to speed up the satis run if you have to deal with a lot of repos. In our case Jenkins will trigger the satis build after a new release was created. Since we have a lot packages (with a lot of versions) in our local satis repo, I was looking for a way to speed up the satis run.

[stof]

shouldn't this be an array argument to allow specifying several packages, like for composer update ?

[shochdoerfer]

Valid point. In our case I only needed one package but obviously this could be extended to allow multiple packages as filters.

[stof]

your filtered package seems to be missing the exclusion of AliasPackage and stability

[shochdoerfer]

@stof thanks for pointing out. Will add that as well.

[shochdoerfer]

When can I expect the PR to be merged in? Or is some more work needed on my side?

[adamlc]

👍 +1 on this.

We only really want Satis to update the individual projects that have passed the tests on our CI server.

[akimsko]

This would be a much welcomed feature. Anything in the way of merging this?

[auipga]

👍 +2 on this.

[jakeasmith]

👍 this would be fantastic

[ralouphie]

+1!

[Burgov]

Big 👍!

[maxime-pasquier]

I actually deal with more than 200 repositories and this will be helpfull.

But I see that the most time past during the build is for building packages.json (if no newer package are found) Did the pull request fix that ?

[shochdoerfer]

@Kmelia That is a good question. Honestly I do not know. We only deal with 10ish repos. Would you mind testing it in your setup?

[maxime-pasquier]

I patch my file vendor/composer/satis/src/Composer/Satis/Command/BuildCommand.php and I have run with just one package : but my packages.json did not keep the others packages. Is that normal ?

[naderman]

Indeed just looking at the code of this it would generate a packages.json only for the filtered packages, not update an existing one with the filtered package info?

[shochdoerfer]

@Kmelia @naderman Thanks for pointing out. Seems I misinterpreted the results. Will work on the update part in the next couple of days.

[akimsko]

@shochdoerfer I added a PR to your fork/branch that should solve this, and some other issues.
https://github.com/bitExpert/satis/pull/1

[shochdoerfer]

Thanks to @akimsko the packages.json file gets now generated correctly as well as the html file. @Kmelia Would you mind testing the patch in your setup?

[naderman]

Unfortunately I recently merged a change to use includes in packages.json so I think this doesn't quite work correctly with that. Can you rebase onto latest master and adapt this branch?

[shochdoerfer]

@naderman I merged in the changes from master this morning and changed the code to make it work with it.

[naderman]

Oh sorry, I missed that :)

[mpdude]

I've been playing with this and was hoping that a such a single-package build would only fetch and re-scan the vcs repository for the given package.

In fact, the console output shows that satis iterates over all repositories again.

Am I missing anything or did I get this feature wrong?

[mpdude]

BuildCommand::selectPackages() will as a first step do

         $repos = $composer->getRepositoryManager()->getRepositories();
          $pool = new Pool($minimumStability);
          foreach ($repos as $repo) {
              try {
                  $pool->addRepository($repo);
              } ...
          }

... which will pull and re-scan all defined VcsRepositories even before the filter is applied.

[akimsko]

Well no, you didnt get it wrong. But it kinda only solved half the problem - the actual package fetching.

TMK you can't select a repo based on package. The information just isn't available. You can only ask if the repo contains a specific package, wich potentially scans all repos to find one specific package.

I guess you could store which repo the package came from (in packages.json or somewhere else), and read from that. But in theory a package can change repo (and maybe even be split across multiple repos in different versions?). So a full scan is the only way to be sure you get all versions of the package.

Of course it could fall back to a full scan, if it can no longer find the package in the last repo it got it from. That would be fairly safe, and very rarely result in a full scan. But it would possibly create some edge cases where new versions wouldn't be found, for instance if the package moved repo, but didnt get removed from the old one.

[mpdude]

Ok, trying to get this straight...

This PR allows to update selectively by providing a package name. It will then re-scan all configured repositories, figure out which versions of the package(s) are available in all of them and then dumpDownload() only versions of this package.

In contrary, PR #110 together with jkufner#1 takes a repo URL as parameter and only re-scans this given repo for available packages. It then merges that information with the full set of packages which is kept serialized in a cache.

What I am aiming for are quick updates by mean of a webhook, as in #40. The URL-based "scan only one repo"-approach seems more fit here.

[mpdude]

Looking at the code more closely:

• The new $packagesFilter argument in selectPackages serves to limit the list of packages available for dumping (line 154).
• Immediately afterwards (line 159ff) we need to "heal" this filtered package list by re-loading dumped package information.
• From that point on, nothing differs in behaviour.

So - wouldn't it be easier to apply the filtering in dumpDownloads only? That would make a much simpler implementation of the same behaviour IMO.

[mpdude]

@shochdoerfer could you please try to explain how this filter is supposed to work together with the require and require-all configuration settings, require-depenencies and require-dev-dependencies?

Thanks!

[shochdoerfer]

@mpdude is my comment in #184 enough? Should the discussion continue there?