Reactive charm built with charmcraft 3.x is missing options from the included layers and is broken

[lgp171188]

Bug Description

We, the Launchpad team, have a bunch of reactive charms that are built using Launchpad charm recipes. Since we did not pin the charmcraft channel to be used for the builds, they are now built with charmcraft 3.x. But the resulting charm files are missing many/all options from the layers included in the charm and are broken. Building the charm with charmcraft 2.x does produce a proper charm that has all the options present.

To Reproduce

https://api.charmhub.io/api/v1/charms/download/wf1A4OyHbIxzKmTWna89ZaDrH618Ft6p_517.charm is the launchpad-appserver charm built using charmcraft 3.x and that is broken. https://api.charmhub.io/api/v1/charms/download/wf1A4OyHbIxzKmTWna89ZaDrH618Ft6p_518.charm is the version that has the exact same charm code but built using charmcraft 2.x and it is not broken.

Environment

These charms are built on the Launchpad builders using charm recipes. But the issue is also reproducible with local charm builds on Ubuntu 24.04 with charmcraft 3.x, whereas 2.x works okay.

charmcraft.yaml

# https://git.launchpad.net/launchpad/tree/charm/launchpad-appserver/charmcraft.yaml
type: charm
bases:
  - build-on:
    - name: ubuntu
      channel: "20.04"
      architectures: [amd64]
    run-on:
    - name: ubuntu
      channel: "20.04"
      architectures: [amd64]
parts:
  charm-wheels:
    source: https://git.launchpad.net/~ubuntuone-hackers/ols-charm-deps/+git/wheels
    source-commit: "42c89d9c66dbe137139b047fd54aed49b66d1a5e"
    source-submodules: []
    source-type: git
    plugin: dump
    organize:
      "*": charm-wheels/
    prime:
      - "-charm-wheels"
  ols-layers:
    source: https://git.launchpad.net/ols-charm-deps
    source-commit: "9c59a9804f1f40e2a74be7dac9bf18a655a7864f"
    source-submodules: []
    source-type: git
    plugin: dump
    organize:
      "*": layers/
    stage:
      - layers
    prime:
      - "-layers"
  launchpad-layers:
    after:
      - ols-layers
    source: https://git.launchpad.net/launchpad-layers
    source-commit: "910dfb76754add5ed032a243c6081ee6a6b11256"
    source-submodules: []
    source-type: git
    plugin: dump
    organize:
      apache-vhost-config: layers/interface/apache-vhost-config
      launchpad-base: layers/layer/launchpad-base
      launchpad-db: layers/layer/launchpad-db
      launchpad-payload: layers/layer/launchpad-payload
    stage:
      - layers
    prime:
      - "-layers"
  layer-coordinator:
    source: https://git.launchpad.net/layer-coordinator
    source-commit: "fa27fc93e0b08000963e83a6bfe49812d890dfcf"
    source-submodules: []
    source-type: git
    plugin: dump
    organize:
      "*": layers/layer/coordinator/
    stage:
      - layers
    prime:
      - "-layers"
  charm:
    after:
      - charm-wheels
      - launchpad-layers
      - layer-coordinator
    source: .
    plugin: reactive
    build-snaps: [charm]
    build-packages: [libpq-dev, python3-dev]
    build-environment:
      - CHARM_LAYERS_DIR: $CRAFT_STAGE/layers/layer
      - CHARM_INTERFACES_DIR: $CRAFT_STAGE/layers/interface
      - PIP_NO_INDEX: "true"
      - PIP_FIND_LINKS: $CRAFT_STAGE/charm-wheels
    reactive-charm-build-arguments: [--binary-wheels-from-source]

Relevant log output

https://launchpadlibrarian.net/743876575/buildlog_charm_ubuntu_focal_amd64_launchpad-appserver_BUILDING.txt.gz has the build logs for the charmcraft 3.x build. https://launchpadlibrarian.net/743923593/buildlog_charm_ubuntu_focal_amd64_launchpad-appserver_BUILDING.txt.gz has the build logs for the charmcraft 2.x build. If needed, I can collect logs from a local build and share them too.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3247.

This message was autogenerated

[lengau]

Thanks for the report! It looks like this is because the config.yaml file gets overwritten.

[lgp171188]

@lengau, is this issue fixed? Asking because the linked PR mentions a fix for actions.yaml getting overwritten. But in this case, it was config.yaml that had the issue?