Set the last three digit of the AMP version to be the number of cherry-picks in the branch

[danielrozenberg]

Implements and closes #27793

[rsimha]

Thanks for embarking on this venture, @danielrozenberg 😃

A few comments below, and one suggestion to make versioning more predictable and eliminate the flag. Let me know what you think.

[rsimha]

Appreciate you patiently dealing with all these review comments, @danielrozenberg. LGTM.

PS: Remember to modify the PR title to say "last three digits" instead of "last digit" 😃

[danielrozenberg]

@mdmower I would love an approval from you before I merge this!

[mdmower]

@mdmower I would love an approval from you before I merge this!

@danielrozenberg I appreciate the consideration! I think the idea is sound, but I have the same concern that @rsimha mentioned here. He seems to consider it a non-issue, so I still have to reconcile these results for myself:

$ git fetch origin
$ git checkout master
$ git reset --hard 2001232119410 # an old tag that exists on master
$ git checkout 2003262059300 # a recent tag with a cherry pick
$ git cherry master | grep '-'
(no results)
$ git cherry origin/master
- (hash)

To me this looks like an out of date local master branch would result in an inconsistent version compared to an up-to-date master. I might be able to convince myself this isn't a problem this evening (after 5pm PDT), but I need to do paid-job work until then.

[danielrozenberg]

No rush!

[mdmower]

@danielrozenberg @rsimha I haven't resolved the dilemma I mentioned in my previous comment. Below is a simplified version. The issue I'm seeing is that an out of date local master branch could result in a different cherry-pick count if the cherry-picks come from the upstream master. The reason for the discrepancy is that this PR references the local master branch when evaluating git cherry master.

$ git fetch origin master:master
$ git checkout master
$ git reset --hard origin/master~1 # local master is 1 commit behind upstream
$ git tag cptest # tag marks where release was cut (1 commit behind upstream)
$ git checkout cptest
$ git cherry-pick origin/master
$ git cherry master
+ 721981a51b52dd83ae8c6e0f2b3af09f290df43d
$ git cherry origin/master
- 721981a51b52dd83ae8c6e0f2b3af09f290df43d

In other words, the build system would identify zero cherry-picks using git cherry master, when it should identify one cherry-pick. git cherry origin/master correctly identifies one cherry-pick, which makes sense... that's where the cherry-pick onto tag cptest came from.

If you agree this is a concern, then you could switch to running git cherry against the latest remote master:

git cherry "$(git config branch.master.remote)/master"

[danielrozenberg]

This is an amazing deep dive into the different ways one can use Git :)

I think this is mostly a non-issue, as I doubt many people will bother doing a fetch from origin/master without fast-forwarding their local master branch, then create a tag, and then cherry pick from origin/master. Even if they do, in that case their local version number will be the time of their cherry-pick, which isn't a total failure, as it only affects their local workspace.

We enforce (both through automation and through due-process) an invariant that would prevent this from affecting actual releases - we reject releases that would return any + commits on git cherry master (or in a human readable format, the invariant is "reject a release that contains any commits that did not come from master)"

I'll merge this PR now so that nightly builds starting next week will use this new format, we can always tweak the finer details later if you still think this is an edge case that should be addressed

[mdmower]

I think this is mostly a non-issue, as I doubt many people will bother doing a fetch from origin/master without fast-forwarding their local master branch, then create a tag, and then cherry pick from origin/master

Just to clarify, I was providing steps for reproducing the issue. They not all necessary in practice. All that's necessary is for someone to run git fetch origin. Then, if they check out a recent release with cherry-picks (e.g. 2004240001480) and build it, it is possible to end up with a different cherry-pick count because their local master was never updated.

EDIT: In fact, here's a real world example:

Simulate a stale master branch

$ git checkout master
$ git reset hard d94fa740e

Checkout a release tag and count the cherry-picks

$ git fetch origin
$ git checkout 2004240001480
$ git cherry master
- (hash)
+ (hash)

In other words, because master is out of date, release 2004240001480 is identified as having zero cherry-picks since latest commit appears like it's directly committed instead of cherry-picked. Simply opting for git cherry "$(git config branch.master.remote)/master" (can break into two steps for Windows compatibility) would take care of this issue.

[danielrozenberg]

I see... Though I don't know why someone would do that :)
In any case, this can only affect your local workspace - if you mess around with git you can always break stuff - we're trying to protect against: release numbers going wrong, and AMP developers not being able to run gulp build because they messed around with with - what AMP version number they get on their local machine rarely matters

[jridgewell]

I'm with @mdmower here, this can cause all kinds of problems. For instance, I don't frequently update my local master. Instead, I do git checkout -b BRANCH upstream/master, which creates my new branches directly from whatever ampproject/amphtml's master is currently on.

As, is, I will always return the wrong number of cherry-picks.

[jridgewell]

Per #27848 (comment), I think this should be reverted.

[jridgewell]

This seems to be the only caller. Why not just remove the %S in the formatted time?

[mdmower]

#27848 (comment)

[danielrozenberg]

Per #27848 (comment), I think this should be reverted.

@jridgewell does it really matter to you what AMP version number shows up when you're doing local development? My main concern with this PR was fixing version numbers for releases, while making sure that local development will continue working uninterrupted (i.e., no errors or completely incomprehensible version numbers)

[mdmower]

I'm hopeful that #28035 will alleviate concern and this commit can be left in-place (no revert necessary).

[jridgewell]

My main concern with this PR was fixing version numbers for releases, while making sure that local development will continue working uninterrupted

This isn't for local development. Our cherry-pick process boils down to:

git clone git@github.com:ampproject/amphtml.git ampproject/amphtml
cd ampproject/amphtml
git checkout -b amp-release-2020-04-27 ${AMP_VERSION}
git cherry-pick -x ${SHA}
git push --set-upstream origin amp-release-2020-04-27

If my local master is out of date, then we've got issues when creating the official release.

[danielrozenberg]

That's why the first line there is git clone, so you'll be working with a fresh copy ;)
@estherkim is working on automating the cherry-picking process directly using GitHub's API, so this would not be a major concern going forward

[jridgewell]

Why would I clone the repo if I already have it? We should add an explicit step to git checkout -B master upstream/master, since it's now a requirement.

[jridgewell]

I actually don't understand why the cherry-pick logic is looking at master at all. Shouldn't we be diffing against the VERSION tag that we branched from?

[estherkim]

It guarantees that you have the cherry pick commit, which is usually a recently merged PR.

[jridgewell]

Can you explain what you mean?

[estherkim]

Hmm well I'm not following the "diffing against the VERSION tag" part. We create a new branch based off the VERSION tag, then apply a commit from master on top of that new branch.

[rsimha]

@jridgewell sorry to pile on to this discussion, but can you summarize your main objection to this PR? (Is the problem the originally stated goal? Or is it the master comparison logic?)

I actually don't understand why the cherry-pick logic is looking at master at all. Shouldn't we be diffing against the VERSION tag that we branched from?

In order to identify which commits were cherry-picked from master to a release branch, we need knowledge of a) the release branch itself, b) the point at which it branched off from master, and c) the up-to-date master branch.

• Knowledge of a) and b) is trivial when you are on the release branch.
• Knowledge of c) requires one of these two things:
  1. Update local master during the CP process (Set the last three digit of the AMP version to be the number of cherry-picks in the branch #27848 (comment))
  2. Compare commits in the CP branch against upstream/master instead of local master (Set the last three digit of the AMP version to be the number of cherry-picks in the branch #27848 (comment))

See the example below.