Zero diff changes

Submitters responsibilities for submitting a PR

Step 0. Start by opening an issue (not a pull request)

Assign the issue to the GCM team leader and only proceed with a PR when the team leader has approved. It is anticipated that some features will be delayed over significant periods of time while other priorities are addressed in the main development stream.

Step 1. Update your feature branch from develop

Option A: If you have never shared your feature branch with anyone, and have not yet pushed it to GitHub

Use git rebase

% git checkout develop
% git pull origin
% git checkout <feature-branch>
% git rebase develop

If there are conflicts, you need to resolve them and commit. (See ????)

Option B: Otherwise ...

Use git merge

% git checkout develop
% git pull origin 
% git checkout <feature-branch>
% git merge develop

If there are conflicts, you need to resolve them and commit. (See ???)

Step 2. Verify zero-diff nature

Perform the following sequence of tests. If a test fails, do not waste resources on subsequent tests until the issue is addressed. For more details on how to run these tests see this page: https://github.com/GEOS-ESM/GEOSgcm_GridComp/wiki/How-to-Run-GEOS-Zero-Diff-Tests

1. 1-day free run (AMIP) with both baseline and feature executables
   Verify all checkpoint/restart files between baseline and feature experiments are identical.
2. 1-day REPLAY with both baseline and feature executables
   Verify all checkpoint/restart files between baseline and feature experiments are identical.
3. 1-day zero-increment REPLAY with feature executable
   Verify all checkpoint/restart files between baseline and feature experiments are identical. This must also be identical to the feature AMIP run in step 1.
4. Run regress script (regress/gcm_regress.j)
   Verify "success" status from regress job

Step 3. Create a PR and check that there are no merge conflicts.

1. If there are merge conflicts, fix them on the feature branch and repeat step 2.
2. Push your feature branch to github if you have not already.
3. Create the PR and start the subject with:

Fixes #xxx

4. Add the "0-diff" label to your PR

Note: The head of the develop branch may have changed by the time the gatekeepers get to your PR. Usually this will not affect zero-diff changes, but they may ask you to merge the latest develop into your feature branch and repeat the verification steps above.

Approval & Merge process

0. Automated tests

These are currently minimal, but we expect to implement aggressive testing in the future. Until automated tests pass, CODEOWNERs and Gatekeepers will generally ignore the PR.

1. CODEOWNERs

Each CODEOWNER team will will inspect relevant code changes to ensure that the changes appear sensible and that implementation quality is acceptable. CODEOWNERs may request changes. If nontrivial changes are required, the zero-diff verifications above will need to be repeated. (And the automated tests will run again.)

2. Gatekeepers

The gatekeeper team will perform the following sequence of tests in order. Once a given test fails, subsequent test will not be performed. Requestor will be notified of issues vi the PR.

1. 1-day free run (AMIP) with both baseline and feature executables
   Verify all checkpoint/restart files between baseline and feature experiments are identical
2. 1-day REPLAY with both baseline and feature executables
   Verify all checkpoint/restart files between baseline and feature experiments are identical
3. 1-day zero-increment REPLAY with feature executable
   Verify all checkpoint/restart files between baseline and feature experiments are identical
4. Run regress script (regress/gcm_regress.j)
   Verify "success" status from regress job

3. Gatekeeper approves the PR in GitHub

If all tests pass, the gatekeeper team performs the following sequence

1. Click merge button
2. Click confirm button
3. Click delete branch button (deletes feature branch)