Builds broken on github, fails at drupal:composer:production step

[tess-ten7]

Starting this week, multiple sites no longer build on Github. The failure occurs when we call ddev task build, which in turn, calls drupal:composer:production. This step fails with the following consistent signature:

[drupal:composer:production] Gathering patches for dependencies. This might take a minute.
[drupal:composer:production]   - Installing drupal/core (10.3.2): Extracting archive
[drupal:composer:production]   0/80 [>---------------------------]   0%
[drupal:composer:production]  10/80 [===>------------------------]  12%
[drupal:composer:production]  20/80 [=======>--------------------]  25%
[drupal:composer:production]  30/80 [==========>-----------------]  37%
[drupal:composer:production]  40/80 [==============>-------------]  50%
[drupal:composer:production]  50/80 [=================>----------]  62%
[drupal:composer:production]  60/80 [=====================>------]  75%Deleting web/modules/contrib/stage_file_proxy - deleted
[drupal:composer:production] 
[drupal:composer:production]  70/80 [========================>---]  87%
[drupal:composer:production] In Application.php line 423:
[drupal:composer:production]                                                         
[drupal:composer:production]   Class "Composer\Console\GithubActionError" not found  
[drupal:composer:production]                                                         
[drupal:composer:production] 
[drupal:composer:production] install [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--download-only] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [--audit] [--audit-format AUDIT-FORMAT] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>...]
[drupal:composer:production] 

Our build task is fairly simple:

  build:prod:
    desc: "Builds the project for production"
    deps:
      - drupal:composer:production
      - assets:prod

If we edit the workflow file and replace this:

      - name: Build Project
        env:
          directory: /tmp/release
        run: |
          ddev composer install
          ddev task build
          ddev task snapshot:directory directory=/tmp/release

with

      - name: Build Project
        env:
          directory: /tmp/release
        run: |
          ddev composer install --no-dev --optimize-autoloader
          ddev task assets:prod
          ddev task snapshot:directory directory=/tmp/release

the build completes without issue or other changes to the repository. This is curious, because we lifted that composer command whole cloth from vendor/lullabot/drainpipe/tasks/drupal.yml which in the repository(s) was:

  composer:production:
    desc: Install composer dependencies without devDependencies
    cmds:
      - composer install --no-dev --optimize-autoloader
    sources:
      - composer.json
      - composer.lock
    generates:
      - ./vendor/composer/installed.json
      - ./vendor/autoload.php
    status:
      - >
        test -f ./vendor/composer/installed.json && grep -q '"dev": false' ./vendor/composer/installed.json

[justafish]

@tess-ten7 if the step is changed to this does it fail?

      - name: Build Project
        env:
          directory: /tmp/release
        run: |
          ddev composer install
          ddev composer install --no-dev --optimize-autoloader
          ddev task assets:prod
          ddev task snapshot:directory directory=/tmp/release

[tess-ten7]

@justafish From what I can tell, no. I also ran into this locally on DDEV which allowed me a moment to poke at it more carefully.

It's apparently a path issue with the task command:

$ ddev task build                                                                                                                                                      feature/MCOMBSUP-211/trashWorkflow|●63✚3⚑4
[drupal:composer:production] PHP Warning:  include(/var/www/html/vendor/bin/../composer/composer/bin/composer): Failed to open stream: No such file or directory in /var/www/html/vendor/bin/composer on line 119
[drupal:composer:production] 
[drupal:composer:production] Warning: include(/var/www/html/vendor/bin/../composer/composer/bin/composer): Failed to open stream: No such file or directory in /var/www/html/vendor/bin/composer on line 119
[drupal:composer:production] PHP Warning:  include(): Failed opening '/var/www/html/vendor/bin/../composer/composer/bin/composer' for inclusion (include_path='.:/usr/share/php') in /var/www/html/vendor/bin/composer on line 119
[drupal:composer:production] 
[drupal:composer:production] Warning: include(): Failed opening '/var/www/html/vendor/bin/../composer/composer/bin/composer' for inclusion (include_path='.:/usr/share/php') in /var/www/html/vendor/bin/composer on line 119

Apparently it's trying to run composer from /var/www/html/vendor/bin instead of /usr/local/bin/composer which is provided by DDEV. This explains why modifying the workflow file worked around the issue, as task was removed and the correct composer version in path was used.

I also found that you can modify the Taskfile.yml in the project replacing the drupal:composer:* steps with fully pathed composer commands and that worked around the issue as well.

[deviantintegral]

Apparently it's trying to run composer from /var/www/html/vendor/bin

Ooo, I just had a related discussion to this in the ddev queue:

• docs: Show how to use a post-start hook to set the composer version ddev/ddev#6688 (comment)
• docs: Show how to use a post-start hook to set the composer version ddev/ddev#6688 (comment)

Is there a reason your project has to require composer? It's typically only needed for libraries that are using the composer API and can cause problems. This also came up in relation to project browser: https://www.drupal.org/project/drupal/issues/3243899

My guess is if you remove the dependency on composer/composer everything will clear itself up!

[tess-ten7]

About that...

In BaseDependencyCommand.php line 103:
                                                              
  Could not find package "composer/composer" in your project  
                                                              

depends [-r|--recursive] [-t|--tree] [--locked] [--] <package>

[deviantintegral]

🤔 then how do you have composer in vendor/bin? I just looked in a somewhat complex project and it doesn't have it. Do you have something that is symlinking it or something?

[deviantintegral]

To clarify: The only time I've seen errors like the above is when composer is a project dependency, and composer is getting upgraded so the classes and files are unexpectedly missing in vendor/bin.

[tess-ten7]

I'm not sure either. The thing is, the failure occurs because something is expecting there to be a composer binary under vendor/bin, but it's not there because we don't touch the composer binary at all.

[deviantintegral]

Since these all run in ddev, you should be able to walk through it by hand locally. Best to do with a fresh checkout or an aggressive git clean... to replicate a CI environment.

I suppose you could also add a whole bunch of ls vendor/bin commands to at least see if it ever changes. But that won't help if it never exists even temporarily.

Otherwise, you could try something like https://github.com/mxschmitt/action-tmate and then manually run each step in the workflow so you can inspect things.