Release Process

[MattHJensen]

The goal of this issue is to formalize a process for releasing new taxcalc, og-usa, b-tax, and taxpuf packages to ospc.org. More fundamentally, this is an issue about how to maintain compatibility across these open source projects.

Below is one proposal that I designed to accomplish the relationship between upstream and downstream packages that I suggested in PSLmodels/Tax-Calculator#1368 (comment).

The proposal below has significant implications for the contributors -- and especially core maintainers -- for all webapp-public associated repositories. It would require explicitly assigning responsibilities to project members from B-Tax, OG-USA, Tax-Calculator, Webapp-Public, and TaxData. It will also require a new "policybrains-modelers" mailing list.

Proposal

1. Upstream project (UP) tags a new release with changelog.
   • Responsibility: Assigned UP contributor
   • Should test locally first against latest release of all projects further upstream.
   • In the case of an independent webapp-public update, release is not tagged, but commit to be released is identified.
2. Build new conda package and release in the main channel.
   • Responsibility: Assigned UP contributor
   • Should make and test a dev release first, but don't need to involve other projects.
   • N/A for independent webapp-public updates.
3. Email policybrains-modelers mailing list w/ package name and changelog
   • Responsibility: Assigned UP contributor
4. Test webapp-public with new UP package and existing versions of other repos: taxcalc, b-tax, og-usa, webapp-public
   • Responsibility: @hdoupe
   • Testing steps should be described in more detail in testing documentation.
   • Note that projects other than webapp-public that are dependent on the new package should be independently testing during this period as well.
5. If failure, email policybrains-modelers mailing list with a description of the failure.
   • Responsibility: @hdoupe
   • If failure was caught through manual inspection, add a new failing test to the webapp-public test suite.
6. Any project that could be causing the failure: run test suite, add new failing test if none are already failing, fix incompatibility, and release new package, starting with step 1 from this document.
   • Responsibility: Assigned UP contributor.
   • UP assignee, email back to list w link to issue.
7. Once all packages are updated and everything appears to work, deploy webapp-public test app.
   • Responsibility: @brittainhard training @hdoupe
8. Email the policybrains-modelers list that everything is online at the test app for the next 1 full business day
   • Responsibility: @hdoupe
9. Anyone who wants to review, reviews.
   • Recommended, anyone who contributed to a change incorporated in the new package(s)
   • An assigned contributor tests. [Need a document that describes what the tests should be.]
   • Write back to mailing list with any problems (return to step 4).
10. Deploy to the main app
    • Responsibility: @brittainhard (screen share w/ @hdoupe)

Assigned UP Contributors:
- Tax-Calculator: @martinholmer or @MattHJensen
- OG-USA: @rickecon or @jdebacker
- Webapp-public: @brittainhard
- B-Tax: @jdebacker
- TaxData: @andersonfrailey

Notes:

• Downstream projects are responsible for fixing incompatibilities w/ upstream projects, but upstream contributors should generally be considerate and as helpful as possible to downstream projects
• Projects should be pinned to use >= latest compatible version.

cc:
Webapp-Public: @PeterDSteinberg @jbcrail @brittainhard @talumbau @enelson1995
Tax-Calculator: @martinholmer @feenberg @Amy-Xu @talumbau
OG-USA: @rickecon @kerkphil @jdebacker
B-Tax: @jdebacker
TaxData: @Amy-Xu @andersonfrailey @martinholmer
Misc: @zrisher @econ02 @GoFroggyRun @hdoupe @codykallen

[martinholmer]

The release process proposed in #560 is a big improvement over the current process. In particular, I think it will improve everybody's working situation to decentralize releases of the different OSPC-sponsored projects. There are too many problems caused by trying to closely coordinate releases of the different projects.

[martinholmer]

@MattHJensen proposed:

1. Upstream project (UP) tags a new release with changelog.

• Responsibility: an assigned UP contributor
• Should test locally first against latest release of all projects further upstream.

2. Build new conda package and release in the main channel.

• Responsibility: an assigned UP contributor
• Should make and test a dev release first, but don't need to involve other projects.

I'd be interested in learning more about the logistics of these first two steps.

Is the new release specified interactively on GitHub after clicking on releases?

With respect to the second step, I know how to create a conda package and install it locally, but I don't know how to upload to www.anaconda.org/ospc.

[GoFroggyRun]

I have a quick comment regarding test dev release.

In particular, @MattHJensen proposed:

1. Upstream project (UP) tags a new release with changelog.

• Responsibility: an assigned UP contributor
• Should test locally first against latest release of all projects further upstream.

2. Build new conda package and release in the main channel.

• Responsibility: an assigned UP contributor
• Should make and test a dev release first, but don't need to involve other projects.
  ...

4. Test webapp-public with new UP package and existing versions of other repos: taxcalc, b-tax, og-usa, webapp-public

• Responsibility: an assigned webapp-public contributor
• Testing steps should be described in more detail in testing documentation.

5. If failure, email policybrains-modelers mailing list with a description of the failure.

• Responsibility: an assigned webapp-public contributor
• If failure was caught through manual inspection, add a new failing test to the webapp-public test suite.

Since we are introducing the dev release for the purpose of testing, we don't necessarily have to treat it as the last buffer before an official release. More specifically, we could, if we'd like, release a dev package whenever significant PRs/changes are being merged.

In this case, testing on web-apps are able to happen sooner while others are working on PRs that would be included, but without having impacts on aggregate results, in the future release. Moreover, having more frequent dev releases as well as web-app testings would be helpful to breakdown potential bug, if any.

Adopting multi stages testing is costly. Before, however, having thorough test suites for web-app, it can help us tracking down potential issues within fairly "large" single release.

[martinholmer]

@GoFroggyRun said in webapp-public issue #560:

Since we are introducing the dev release for the purpose of testing, we don't necessarily have to treat it as the last buffer before an official release. More specifically, we could, if we'd like, release a dev package whenever significant PRs/changes are being merged.

In this case, testing on web-apps are able to happen sooner while others are working on PRs that would be included, but without having impacts on aggregate results, in the future release. Moreover, having more frequent dev releases as well as web-app testings would be helpful to breakdown potential bug, if any.

Adopting multi stages testing is costly. Before, however, having thorough test suites for web-app, it can help us tracking down potential issues within fairly "large" single release.

Thanks for the discussion of testing, @GoFroggyRun. I'm in full agreement with the first sentence in your last paragraph: Adopting multi-staged testing is costly. So, it seems to me the more cost-effective strategy is to work towards having [a] thorough test suite for webapp-public.

I can contribute code that has been and is now in the Tax-Calculator repo that should be part of any webapp-public test suite. Is there a pytest suite now? If so, what is the code coverage?

@MattHJensen @PeterDSteinberg @brittainhard @andersonfrailey @hdoupe

[PeterDSteinberg]

@MattHJensen @brittainhard @andersonfrailey @GoFroggyRun @jdebacker Sorry it has taken me so long to comment on this issue. I think the Release outlined plan looks good. One thing I will add is that we should modify policybrain-builder so that it can release packages that are:

• Built from the open-source-economics Github organization as it is done now or alternatively from a personal fork of a repo
• Built from a given git branch or tag, but released under a different name, e.g. I want to build Tax-Calculator conda pkg from the "parameters-mods" branch but distribute the package via psteinberg/label/dev on anaconda.org labels and rename the pkg.

I'll assign these issues above to @jbcrail in the policybrain-builders repo (@MattHJensen - please make sure @jbcrail is in the contributors for policybrain-builder). The steps I mention above will make it easier for devs to do intermediate releases during the release process.

[martinholmer]

@PeterDSteinberg said on Thursday, June 29th:

Sorry it has taken me so long to comment on this issue [webapp-public #560]. I think the Release [Process] outlined [here] looks good. One thing I will add is that we should modify policybrain-builder so that it can release packages that are:

• Built from the open-source-economics Github organization as it is done now or alternatively from a personal fork of a repo
• Built from a given git branch or tag, but released under a different name, e.g. I want to build Tax-Calculator conda pkg from the "parameters-mods" branch but distribute the package via psteinberg/label/dev on anaconda.org labels and rename the pkg.

I'll assign these issues above to @jbcrail in the policybrain-builders repo (@MattHJensen - please make sure @jbcrail is in the contributors for policybrain-builder). The steps I mention above will make it easier for dev[eloper]s to do intermediate releases during the release process.

I agree that the Release Process described in issue #560 implies that the policybrain-builder/build_release.sh script needs to be revised, but the nature of the needed revisions are much different than those described above.

The current design of the build_release.sh script is incompatible with the Release Process described in #560. The Release Process is loosely-coupled, in the sense that the various OSPC-sponsored projects (that are located in different GitHub repositories) are assumed to be on their own development path where the timing of enhancements can be very different than the timing of enhancements in other projects. The current version of the build_release.sh script is tightly-coupled, in the sense that it assumes the projects that rely on Tax-Calculator are moving in lock-step with Tax-Calculator.

The whole point of this Release Process is to get away from this tightly-coupled, lock-step release approach.

My guess is that a new version of the build_release.sh script will need to eliminate for BTAX builds and for OGUSA builds the taxcalc pinning logic in the current version of the script and rely instead on the taxcalc version specified in the conda.recipe/meta.yaml file of each project that relies on Tax-Calculator. The developers of those projects can decide when they want to use a new version of the taxcalc package by editing the conda.recipe/meta.yaml file for their project.

@MattHJensen @PeterDSteinberg @jbcrail @rickecon @jdebacker

[MattHJensen]

My guess is that a new version of the build_release.sh script will need to eliminate for BTAX builds and for OGUSA builds the taxcalc pinning logic in the current version of the script and rely instead on the taxcalc version specified in the conda.recipe/meta.yaml file of each project that relies on Tax-Calculator. The developers of those projects can decide when they want to use a new version of the taxcalc package by editing the conda.recipe/meta.yaml file for their project.

A corollary to this is that webapp-public will need to set specific pins as well, rather than always targeting the latest version of every project. This should interact well with @brittainhard's suggestion to version webapp-public.

@PeterDSteinberg @jbcrail @brittainhard

[brittainhard]

@MattHJensen +1 on versioning webapp with package specifications.