x/build/cmd/gerritbot: first "PR has been imported to Gerrit" message doesn't get posted

[dmitshur]

This is a subset or a more specific issue related to #27504.

I'm noticing that the first message that gerritbot is supposed to post (as gopherbot user) is no longer appearing.

Here's an old PR where it appears:

• net/http: optionally add headers on redirect #24004 (comment)

Here are newer PRs where it does not:

• clean: clean mod cache should respect "-n" option. #27471
• clean: clean cache with -n should only print remove commands. #27470

I noticed this happen when I created golang/dl#1. The first message didn't get posted, but the following ones did.

I captured the logs, and I suspect I can see why it's not working.

Logs

Creating the PR:

2018/09/07 20:46:11 Retrieving PR golang/dl#1 from GitHub without an Etag ...
2018/09/07 20:46:11 GitHub: 4967/5000 calls remaining; Reset in 43m9.527324806s
2018/09/07 20:46:11 Processing PR https://github.com/golang/dl/pull/1 ...
2018/09/07 20:46:11 Executing [git clone --bare https://go.googlesource.com/dl /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl]
2018/09/07 20:46:11 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl remote add github git://github.com/golang/dl.git]
2018/09/07 20:46:11 Executing [rm -rf /root/.gerritbot/tmp/worktree_golang_dl_1]
2018/09/07 20:46:11 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl worktree prune]
2018/09/07 20:46:11 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl worktree add /root/.gerritbot/tmp/worktree_golang_dl_1]
2018/09/07 20:46:11 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 fetch origin +master:master]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 fetch github pull/1/head]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 merge-base master FETCH_HEAD]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 diff-tree --always --no-patch --format=%an <%ae> FETCH_HEAD]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 checkout -B golang/dl#1 60e9c3299d84559cae581fe3818300813eb9ba1b]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 merge --squash --no-commit FETCH_HEAD]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 commit --author Dmitri Shuralyov <dmitri@shuralyov.com> -m remove blank line from codereview.cfg

It's not needed. It's not there in other subrepos:

- https://github.com/golang/image/blob/c73c2afc3b812cdd6385de5a50616511c4a3d458/codereview.cfg
- https://github.com/golang/net/blob/161cd47e91fd58ac17490ef4d742dc98bb4cf60e/codereview.cfg

Using this to test PRs for golang/go#26949.

Change-Id: I7152ea2dae1a511928e787598696274cf2b315f9
GitHub-Last-Rev: b41da79638ca03d8baf47b66f1e033fb04337ddb
GitHub-Pull-Request: golang/dl#1
]
2018/09/07 20:46:12 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 push -o nokeycheck origin HEAD:refs/for/master]
2018/09/07 20:46:14 Cleaning up...
2018/09/07 20:46:14 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 checkout master]
2018/09/07 20:46:14 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 branch -D golang/dl#1]
2018/09/07 20:46:14 Executing [rm -rf /root/.gerritbot/tmp/worktree_golang_dl_1]
2018/09/07 20:46:14 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl worktree prune]
2018/09/07 20:46:14 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl branch -D worktree_golang_dl_1]
2018/09/07 20:46:14 processPullRequest: importGerritChangeFromPR(golang/dl#1, nil): could not find change URL in command output: "remote: \rremote: Processing changes: new: 1 (\\)\rremote: Processing changes: new: 1 (|)\rremote: Processing changes: refs: 1, new: 1 (|)\rremote: Processing changes: refs: 1, new: 1 (|)        \rremote: Processing changes: refs: 1, new: 1, done            \nremote: \nremote: SUCCESS        \nremote: \nremote: New Changes:        \nremote:   https://go-review.googlesource.com/c/dl/+/134117 remove blank line from codereview.cfg        \nTo https://go.googlesource.com/dl\n * [new branch]      HEAD -> refs/for/master"

Amending the PR:

2018/09/07 20:49:17 Retrieving PR golang/dl#1 from GitHub using Etag "W/\"71eea34ea5a8c7aeb5c0e16b93315306\"" ...
2018/09/07 20:49:17 GitHub: 4906/5000 calls remaining; Reset in 40m3.395245934s
2018/09/07 20:49:17 Processing PR https://github.com/golang/dl/pull/1 ...
2018/09/07 20:49:17 Executing [rm -rf /root/.gerritbot/tmp/worktree_golang_dl_1]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl worktree prune]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl worktree add /root/.gerritbot/tmp/worktree_golang_dl_1]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 fetch origin +master:master]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 fetch github pull/1/head]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 merge-base master FETCH_HEAD]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 diff-tree --always --no-patch --format=%an <%ae> FETCH_HEAD]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 checkout -B golang/dl#1 60e9c3299d84559cae581fe3818300813eb9ba1b]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 merge --squash --no-commit FETCH_HEAD]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 commit --author Dmitri Shuralyov <dmitri@shuralyov.com> -m dl: remove blank line from codereview.cfg

It's not needed. It's not there in other subrepos:

- https://github.com/golang/image/blob/c73c2afc3b812cdd6385de5a50616511c4a3d458/codereview.cfg
- https://github.com/golang/net/blob/161cd47e91fd58ac17490ef4d742dc98bb4cf60e/codereview.cfg

Using this to test PRs for golang/go#26949.

Change-Id: I7152ea2dae1a511928e787598696274cf2b315f9
GitHub-Last-Rev: b41da79638ca03d8baf47b66f1e033fb04337ddb
GitHub-Pull-Request: golang/dl#1
]
2018/09/07 20:49:17 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 push -o nokeycheck origin HEAD:refs/for/master]
2018/09/07 20:49:19 Cleaning up...
2018/09/07 20:49:19 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 checkout master]
2018/09/07 20:49:19 Executing [git -C /root/.gerritbot/tmp/worktree_golang_dl_1 branch -D golang/dl#1]
2018/09/07 20:49:19 Executing [rm -rf /root/.gerritbot/tmp/worktree_golang_dl_1]
2018/09/07 20:49:19 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl worktree prune]
2018/09/07 20:49:19 Executing [git -C /root/.gerritbot/repos/https:%2F%2Fgo.googlesource.com%2Fdl branch -D worktree_golang_dl_1]
2018/09/07 20:49:19 processPullRequest: importGerritChangeFromPR(golang/dl#1, &{0xc03530b9f0 134117 2018-09-07 20:46:13 +0000 +0000 1 {GitCommit 204de0d4586b1c069f7e928abefc9ae3e580acf3} master 0xc04734efe0 [0xc04734f000] new false [golang/go#26949 golang/dl#1] []}): could not find change URL in command output: "remote: \rremote: Processing changes: updated: 1 (\\)\rremote: Processing changes: refs: 1, updated: 1 (\\)\rremote: Processing changes: refs: 1, updated: 1 (\\)\rremote: Processing changes: refs: 1, updated: 1 (\\)\rremote: Processing changes: refs: 1, updated: 1, done            \nremote: warning: 73715c7: no files changed, message updated        \nremote: \nremote: SUCCESS        \nremote: \nremote: Updated Changes:        \nremote:   https://go-review.googlesource.com/c/dl/+/134117 dl: remove blank line from codereview.cfg        \nremote: \nTo https://go.googlesource.com/dl\n * [new branch]      HEAD -> refs/for/master"

Specifically, see the last line with the "could not find change URL in command output" error:

processPullRequest: importGerritChangeFromPR(golang/dl#1, nil): could not find change URL in command output: "remote: \rremote: Processing changes: new: 1 (\\)\rremote: Processing changes: new: 1 (|)\rremote: Processing changes: refs: 1, new: 1 (|)\rremote: Processing changes: refs: 1, new: 1 (|)        \rremote: Processing changes: refs: 1, new: 1, done            \nremote: \nremote: SUCCESS        \nremote: \nremote: New Changes:        \nremote:   https://go-review.googlesource.com/c/dl/+/134117 remove blank line from codereview.cfg        \nTo https://go.googlesource.com/dl\n * [new branch]      HEAD -> refs/for/master"

I suspect what happened is that the URL format that Gerrit returns has changed from:

https://go-review.googlesource.com/#/c/dl/+/134117

To:

https://go-review.googlesource.com/c/dl/+/134117

And it no longer matches this regexp:

https://github.com/golang/build/blob/a196f5a8e8d347b6b9fe4cd225a3393a77c03214/cmd/gerritbot/gerritbot.go#L532-L533

/cc @andybons

[gopherbot]

Change https://golang.org/cl/134175 mentions this issue: cmd/gerritbot: update gerritChangeRE for new Gerrit output format

[dmitshur]

This is deployed.

I've confirmed it's working. I created a new PR just now, and the "This PR has been imported to Gerrit for code review" comment was posted:

golang/dl#2 (comment)

Also, editing an old PR where the comment wasn't posted earlier causes it to get posted:

golang/dl#1 (comment)

Better late than never!