cci flow run ci_beta AND cci task install_managed_beta DO not use the latest beta

[kayla-blackthorn]

Describe the bug

I am trying to set up a GitHub Actions workflow that creates a new 2gp beta package version and then runs all tests.
I have permissions that need to get added as a post deploy step so I am skipping validation in the first step of the workflow cci task run create_package_version --org dev --package_type Managed --skip_validation true

I can see in the logs that the package version is created successfully:
[Success]: Package creation successful
Created package version:
Package2 Id: 0HoVX00000000850AA
Package2Version Id: 05iVX000000DUgDYAW
SubscriberPackageVersion Id: 04tVX0000000BOTYA2
Version Number: 0.1.0.3
Dependencies: []

But then when I run ci_beta it uses Version Number 1.27 which is the last major (production) release created for the package in the packaging org. (We still use 1gp for prod releases)

I tried running each task individually and then explicitly setting version: latest_beta in the step install_managed_beta and that still results in version 1.27 being used.

I then also tried explicitly setting the version to ^^create_package_version.version_number but when I do that I receive this error:
Installing Blackthorn Base 0.1.0.3
Pending
[Pending]: next check in 1 seconds
[02/07/24 20:37:06] [Done]
[02/07/24 20:37:07] [Failed]: Update of InstalledPackage bt_base: Error:
InstalledPackage version number : 0.1.0.3 does not exist!

So this is a two part issue:

Part 1 - Why do ci_beta and install_managed_beta use the latest major release instead of the latest beta as per documentation?

Part 2 - Why when I set the version explicitly to 0.1.0.3 does it say that version does not exist when I can clearly see in the logs or by running sf package version list that it does?

Reproduction steps

1. Run salesforce-nightly-build.yml
2. create_package_version task runs as part of yml workflow
3. Verify that new beta version created
4. Execute ci_beta as part of workflow
5. Verify that version 1.27 is used and not latest beta
6. Retry and edit workflow to separate individual tasks
7. Set install_managed_beta task to use ^^create_package_version.version_number
8. Verify that an error is received due to version not existing

Your CumulusCI and Python versions

CumulusCI version: 3.84.0
Python version: 3.10.4

Operating System

macOS

Windows environment

No response

CumulusCI installation method

pip

Error Gist

No response

Additional information

No response

[jlantz]

Are you using the release_2gp_beta flow? That flow has extra steps to publish the release artifacts as tags and a release in the repository. Those artifacts are what ci_beta is looking for. If it doesn't find a beta since the last production, it uses the last production.

The documentation for releasing 2GP packages walks through the workflow built into CumulusCI for creating betas and publishing the artifacts:
https://cumulusci.readthedocs.io/en/stable/managed-2gp.html#create-a-beta-version

I'm not sure this is a bug. It may be a documentation usability issue, but the ci_beta flow's job is to install the latest beta published by either release_beta for 1GP or release_2gp_beta for 2GP.

[kayla-blackthorn]

@jlantz thank you for clarifying. I was extracting the create_package_version step from the release_2gp_beta flow because I didn't want to create a release/tags in GitHub for every 2gp beta version since I am using this for nightly builds.

If the goal here is to deploy and package the contents of our main branch each night and make sure no tests fail is release_2gp_beta followed by ci_beta the approach that you'd recommend? Or do you think using one of the feature flows would make more sense?

[jlantz]

CumulusCI's default process, which I'd also recommend, is to create beta versions whenever a pull request is merged using release_2gp_beta and then running ci_beta to test the beta in a new scratch org. The idea is to test any changes to branch staged on the release train, generally main. The beta tags are prefixed differently and the releases are set as prerelease versions so GitHub will highlight the latest production release.

Note that release_2gp_beta does include github_automerge_main which will merge the newly built commit down to all feature branches that don't have it. You can disable but still keep the tag and release creation that with:

flows:
    release_2gp_beta:
        steps:
            4:
                task: None

Here's the reusable workflow for 2GP beta builds I've created in D2X:
https://github.com/muselab-d2x/d2x/blob/main/.github/workflows/beta-test-2gp.yml

[kayla-blackthorn]

Thank you @jlantz for clarifying the steps and explaining the intention behind the default process. I was curious at one point if the tags/releases might be what the ci_beta flow was counting on existing to find the latest beta. I think it could be helpful to others to clarify that in the documentation. I really appreciate you taking the time to help me with this! I'll go ahead and close this case and let you know if I have any further questions. 😄

[jlantz]

I'll let the CumulusCI team review and close. I'm just a community contributor myself these days ;)

The source for all the docs is under the docs/ directory if you have ideas on where that information could be placed to be most helpful to someone coming from your perspective.