Automate creation of Issue for major release PHP synchronisation #57890

getdave · 2024-01-16T16:05:26Z

What?

Automates the creation of the contents for the Issue that tracks PHP changes (example) that may need to be synchronised to WP Core for a major release.

Why?

For a major release, the Editor Tech Lead needs to create an Issue like this one which details all the PHP changes that may need backport to Core.

This requires manually going through all the key directories in the project that may contain PHP changes and then checking all the PRs that have been merged since the final RC of the previous release. If any PRs contain PHP changes then the PR needs to be added to the Issue in a very specific format.

It also requires organising headings for each seciton and doing a lot of manual formatting.

All this is:

laborious - I'm told it typically requires a full day or more to complete
prone to error - it's easy to make a typo or miss a change

This is why I think it's wide open for automation.

How?

This PR has a script which can be run locally which will

get all commits to trunk in a number of key directories (including /lib, /block-library and /phpunit) since a date you provide.
get the details for each of the commits including the files that were changed
put the commit under the correct heading based on which files were changed
dedupe the result set
output the markdown required to create the Issue

I've made some subjective choices about how things are group and displayed:

~~PRs may be listed only once in the Issue - the exception is PRs which modify PHP Unit tests as I wanted to display a full list of changes that affect the /phpunit directory.~~ Actually PRs are allowed more than once as long as it's across different sections
No more than 3 level of nesting - anything beyond that gets very visually messy. I've grouped PRs that would appear under those subheadings under the third level of nesting.
Certain directories are excluded including e2e-tests and experiments-page.php which are never synchronised - please feel free to suggest additions to that list.
I've elected to avoid the additional network requests required to fetch the details of the PRs associated with each commit. Instead I've used regex to attempt to build the PR URL from the commit message. This isn't 100% reliable but it works in the majority of cases and we have a reasonable fallback in place.

Testing Instructions

Get GH Personal Access Token
Run node bin/generate-php-sync-issue.mjs --token={YOUR_TOKEN_HERE} --since=2023-11-01
Open issueContent.md and copy all contents.
Go to https://github.com/WordPress/gutenberg/issues/new/ and paste contents.
Preview the Issue - DO NOT SUBMIT IT!
Take any given PR in the Issue and check that the files it modified match up with where it is listed in the Issue.

Testing Instructions for Keyboard

Screenshots or screencast

getdave · 2024-01-16T16:08:02Z

bin/generate-php-sync-issue.mjs

+	const content = generateIssueContent( result );
+
+	// Write the Markdown content to a file
+	fs.writeFileSync( nodePath.join( __dirname, 'issueContent.md' ), content );


Currently writes to a file. We could have it automatically create the Issue on Github.

I'm not sure about this, since the generated content will almost certainly need a bit of editing. Might just be easier to write to file and the person generating it can copy/paste/edit before publishing.

Sounds fine to me. It's a very manual script anyway. The overhead of auto-creating an Issue probably isn't worth the effort.

I agree we shouldn't create an issue but we could actually output the content in the console. So the commands become something like yourcommand > issueContent.md if you really want to put the content in a file. That seems more inline with how tools like that work.

getdave · 2024-01-16T16:08:40Z

bin/generate-php-sync-issue.mjs

+		// The paginate method will fetch all pages of results from the API.
+		// We limit the total results because we only fetch commits
+		// since a certain date (via the "since" param).
+		commits = await octokit.paginate( 'GET /repos/{owner}/{repo}/commits', {
+			owner,
+			repo,
+			since,
+			per_page: 100,
+			path,
+		} );


This can generate a fair number of requests

getdave · 2024-01-16T16:21:02Z

bin/generate-php-sync-issue.mjs

+			owner,
+			repo,
+			since,
+			path


Not using path will result in a tonne of commits that have little or no value. So actually it's better to only fetch commits against the paths we know will have PHP files that need to be synced.

getdave · 2024-01-16T16:21:52Z

bin/generate-php-sync-issue.mjs

+		? `https://github.com/WordPress/gutenberg/pull/${ prId }`
+		: `[Commit](${ commit.html_url })`;


If we can't build the PR URL from the commit message, then we'll fall back to the commit URL itself and let GH UI show the link to the PR there.

bin/generate-php-sync-issue.mjs

Mamaduka · 2024-01-17T10:22:37Z

I believe @ramonjd was working on something similar last year.

bin/generate-php-sync-issue.mjs

tellthemachines

Thanks for the PR, this is working pretty well for me!

Regarding the generated content:

the compat/wordpress-6.4 folder can be removed, since whatever is in there is already in core. It exists to provide compatibility for WP 6.3, because we have to support the current and the previous WP versions. Not sure if there's a simple way to automate that as the folder numbers will vary per release - might be easier to just delete it manually.
load.php is plugin-specific so we probably don't need to look at it either.
not sure if this is possible: could we remove some PRs from the list if they have a certain label? Anything with Backport from WordPress Core can safely be ignored.
Would it be worth checking the Needs PHP backport label and including any PRs from there that aren't in the list?

One thing to bear in mind is that we should include all PRs since the last plugin release to be fully included in Beta 1, because anytime after that new features could have been merged that would still need syncing. See my comment on validateSince below 😄

Other than that I think this is looking pretty good! I'm no node expert though so you might want a second opinion on the code itself.

bin/generate-php-sync-issue.mjs

tellthemachines · 2024-01-18T07:12:43Z

bin/generate-php-sync-issue.mjs

+	const content = generateIssueContent( result );
+
+	// Write the Markdown content to a file
+	fs.writeFileSync( nodePath.join( __dirname, 'issueContent.md' ), content );


I'm not sure about this, since the generated content will almost certainly need a bit of editing. Might just be easier to write to file and the person generating it can copy/paste/edit before publishing.

bin/generate-php-sync-issue.mjs

youknowriad

This looks great to me. Some valid remarks above from @tellthemachines but otherwise it's a perfect head start to the backporting process. Nice work everyone.

getdave · 2024-01-18T12:05:47Z

Thanks for review @tellthemachines 👍

the compat/wordpress-6.4 folder can be removed, since whatever is in there is already in core. It exists to provide compatibility for WP 6.3, because we have to support the current and the previous WP versions. Not sure if there's a simple way to automate that as the folder numbers will vary per release - might be easier to just delete it manually.

Currently it will remove the folder for the current stable release but I think we need to make it a whitelist to retain only the folder for the next release and remove all others.

load.php is plugin-specific so we probably don't need to look at it either.

✅ Done

not sure if this is possible: could we remove some PRs from the list if they have a certain label? Anything with Backport from WordPress Core can safely be ignored.

Apparently this label gets removed once the PR has been backported as a way of indicating it's completion status. So instead @youknowriad and I agreed that we would flag any PRs made between Beta 1 and final RC with a special warning.

Would it be worth checking the Needs PHP backport label and including any PRs from there that aren't in the list?

Yes that would be a nice addition.

One thing to bear in mind is that we should include all PRs since the last plugin release to be fully included in Beta 1, because anytime after that new features could have been merged that would still need syncing. See my comment on validateSince below 😄

--since will work with whatever you provide. I've extended the max allowed date.

getdave · 2024-01-18T14:23:15Z

Given that this is not production code we could probably look to merge and iterate on this.

bin/generate-php-sync-issue.mjs

youknowriad · 2024-01-18T14:37:12Z

not sure if this is possible: could we remove some PRs from the list if they have a certain label? Anything with Backport from WordPress Core can safely be ignored.

@getdave "from WordPress Core" is another label and these can be safely ignored regardless of date...

ramonjd · 2024-01-18T20:40:02Z

I believe @ramonjd was working on something similar last year.

Thanks for the ping. I had a local shell script trying to do what this PR's script does, but mine was definitely not as good and detailed as this one, and required too much manual clean up. Thanks for this initiative @getdave!

tellthemachines

LGTM! Small non-blocking typo and comment below. I think we can merge and iterate.

bin/generate-php-sync-issue.mjs

Mamaduka

Excellent work, Dave.

Maybe, as a follow-up, we can document the usage of this script. Here is an example of cherry-picking automation docs - https://developer.wordpress.org/block-editor/contributors/code/release/auto-cherry-picking/.

getdave · 2024-01-19T10:01:52Z

Backport from WordPress Core

✅ I've set up an ignore list for any labels. This can be added to in followups.

getdave · 2024-01-19T10:19:59Z

Just noting I used this script to generate #57959

getdave added 3 commits January 16, 2024 16:03

Initial script

d31097c

Linting fun

7a912c8

Add primitve docs to args

322c046

getdave commented Jan 16, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Outdated Show resolved Hide resolved

getdave commented Jan 16, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Outdated Show resolved Hide resolved

getdave self-assigned this Jan 17, 2024

getdave added the [Type] Project Management Meta-issues related to project management of Gutenberg label Jan 17, 2024

getdave requested review from scruffian, mikachan and tellthemachines January 17, 2024 09:50

getdave changed the title ~~[WIP] Release automation testing~~ Automate creation of Issue for major release PHP synchronisation Jan 17, 2024

getdave requested review from annezazu and priethor January 17, 2024 09:56

Ensure since arg cannot be far into the past

fbbddbf

getdave marked this pull request as ready for review January 17, 2024 10:06

getdave requested review from ntwb, nerrad and ajitbohra as code owners January 17, 2024 10:06

getdave requested a review from Mamaduka January 17, 2024 10:06

Mamaduka reviewed Jan 17, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Outdated Show resolved Hide resolved

getdave added 3 commits January 17, 2024 10:25

Refactor to avoid passing around octokit vars

f949eba

Handle network errors

803baa7

Refactor method names to use “fetch” terminology

c7def9d

getdave requested a review from ramonjd January 17, 2024 10:33

Ignore block-library as these changes are handled automatically

ace8c85

Remove block-library specific handling

71cb3ef

getdave commented Jan 17, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Show resolved Hide resolved

tellthemachines reviewed Jan 18, 2024

View reviewed changes

getdave commented Jan 18, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Outdated Show resolved Hide resolved

youknowriad reviewed Jan 18, 2024

View reviewed changes

getdave added 8 commits January 18, 2024 09:30

Use const to define max nesting level

5e05e7c

Automate ignore of previous release compat dir

780b842

Exclude load.php as it is Plugin specific

a9872a1

Remove ping debugging

d6cdb4f

Prepare for requiring full PR label data

9fe1f10

Flag PRs that may have already been backported

a8cbfef

Fix output of undefined

6bfb96c

Tweak warning

39ec158

mikachan reviewed Jan 18, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Outdated Show resolved Hide resolved

tellthemachines approved these changes Jan 18, 2024

View reviewed changes

bin/generate-php-sync-issue.mjs Outdated Show resolved Hide resolved

Mamaduka approved these changes Jan 19, 2024

View reviewed changes

Ignore PRs with given labels

5bff9d0

getdave added 2 commits January 19, 2024 10:04

Update instructions for new label

cf8e370

Update since arg description

655af32

getdave enabled auto-merge (squash) January 19, 2024 10:10

getdave merged commit b716ae3 into trunk Jan 19, 2024
54 of 55 checks passed

getdave deleted the try/automate-php-synchronisation-issue-template branch January 19, 2024 10:40

github-actions bot added this to the Gutenberg 17.6 milestone Jan 19, 2024

getdave mentioned this pull request Jan 19, 2024

Document new script for creating release PHP synchronisation issue #58000

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automate creation of Issue for major release PHP synchronisation #57890

Automate creation of Issue for major release PHP synchronisation #57890

getdave commented Jan 16, 2024 •

edited

Loading

getdave Jan 16, 2024

tellthemachines Jan 18, 2024

getdave Jan 18, 2024

youknowriad Jan 18, 2024

getdave Jan 16, 2024

getdave Jan 16, 2024

getdave Jan 16, 2024

Mamaduka commented Jan 17, 2024

tellthemachines left a comment

tellthemachines Jan 18, 2024

youknowriad left a comment

getdave commented Jan 18, 2024

getdave commented Jan 18, 2024

youknowriad commented Jan 18, 2024

ramonjd commented Jan 18, 2024 •

edited

Loading

tellthemachines left a comment

Mamaduka left a comment •

edited

Loading

getdave commented Jan 19, 2024

getdave commented Jan 19, 2024

		? `https://github.com/WordPress/gutenberg/pull/${ prId }`
		: `[Commit](${ commit.html_url })`;

Automate creation of Issue for major release PHP synchronisation #57890

Automate creation of Issue for major release PHP synchronisation #57890

Conversation

getdave commented Jan 16, 2024 • edited Loading

What?

Why?

How?

Testing Instructions

Testing Instructions for Keyboard

Screenshots or screencast

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mamaduka commented Jan 17, 2024

tellthemachines left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youknowriad left a comment

Choose a reason for hiding this comment

getdave commented Jan 18, 2024

getdave commented Jan 18, 2024

youknowriad commented Jan 18, 2024

ramonjd commented Jan 18, 2024 • edited Loading

tellthemachines left a comment

Choose a reason for hiding this comment

Mamaduka left a comment • edited Loading

Choose a reason for hiding this comment

getdave commented Jan 19, 2024

getdave commented Jan 19, 2024

getdave commented Jan 16, 2024 •

edited

Loading

ramonjd commented Jan 18, 2024 •

edited

Loading

Mamaduka left a comment •

edited

Loading