Epic: Project migration from classic to beta

[JessicaLucindaCheng]

Dependency

• When the new projects boards are out of Beta (currently not stable enough to create GHAs against)
• Audit for changing labels from size to complexity #2777
• Modifying GitHub actions for label change from size to level  #3109

Details

• "Projects (classic) will be sunset on August 23, 2024."
• 
• November 19, 2024: The REST API for Projects (classic) will be sunset.

Overview

As a member of the Website Team, we need to prepare for the migration of our Project Board from "Projects (classic)" to "Projects (Beta)" so that if "Projects (classic)" is deprecated we are ready to use "Projects (Beta)" with our team. For this issue, we will figure out what needs to be done in order for us to migrate, add those as Action Items below, and complete those Action Items.

Action Items

• Write a back-end issue to revise move-closed-issues.yaml to trigger on all issues on the board that are open and have an assignee so that the actions work on both classic and beta project boards. --> Dev lead is responsible.
• Ask rep Josie about GitHub Kanban automation so that our PRs can go into the correct columns --> Dev lead is responsible. See the note here: Epic: Project migration from classic to beta #3265 (comment)
• For move-closed-issues.yaml, we need to find out if we can change the status of an issue via the API. (Status is the new name of columns in the beta project boards.) --> Dev lead is responsible.
• The winsFormResponse Google Apps Script file in our admin drive has a column id reference as well on line 123. This is responsible for adding a new issue created by a bot to a specific column. Saumil's understanding is that the issue would still be created just won't be added to the project (beta) board. So, we would need to decide next steps for this one.
• Epic Prep for Project Board Migration: GitHub Actions / Workflows Part 1 of 2  #6993 for Pre and Post-Project Migration issues

Notes for emails from GitHub rep Josie

Features related to automations and insights in the new project beta

Those tracks of work are on hold until GA but someone from GitHub might reach out once teams at GitHub are ready to get that track of work started. Can follow up with Josie on timelines for this.

Information on a bug and how to resolve it

If you’re seeing the “Migration complete” modal and you are unable to dismiss it, it is likely that you are not opted in to the feature preview for project migration. GitHub team recently put the entire feature behind a feature preview so if you are not in the feature preview, you will run into issues like this.
Action: If you are using project migration and have not yet opted in to the feature preview, please do so before you continue accessing previously migrated projects.

Resources/Instructions

• Project board migration related session videos with GitHub rep
• Related to issue Change alt text for sdg image in 311 data md file #3091

[JessicaLucindaCheng]

The notes below are copied from #3031 (comment)

Notes from June 15 meeting with Josie from GitHub

• The beta projects have an issue limit of 1200. Currently, we have 1195. Thus, the first thing we need to do is Archive some issues or somehow reduce the number.
• On our current project board, Bonnie discusses how we have cards with filters (by role and complexity) and as info/descriptions at the top of each column. Since cards become draft issues, we need to figure out how we going to have
• filters (by role and complexity)
• the info/description cards (though Bonnie said maybe we could move some of these into the Wiki)
• You can archive one issue at a time (or a whole column)
  • May need to show Bonnie how to archive one issue at a time
• Want to figure out how to evaluate if another person (of a certain role) needs to be recruited to the team by filtering by role and looking at size/pts, and seeing if there are more than 2 months worth of points.
• Even though currently we have both a classic version and a beta version of the project board, they may not be in sync.
• Migration feature to be rolled out soon (maybe in next few days) would enable all members with write-access to be able to migrate the project to beta version.
• If a project is migrated to the beta version second time, the old beta would disappear and get deleted, resulting in loss of all the progress and changes made in the old beta project.
• We were not able to locate column ids or links in the new project beta version and the GitHub team would get back to us on that, but some of our GitHub actions might be impacted because of this.
• Header of the new project beta has different features than the classic project board and there is separate GitHub team responsible for that. So, it may not be part of the migration release.

[JessicaLucindaCheng]

1. For future reference, I have taken a screenshot of the default "Who has access" and "Manage access" settings of the beta project board when we did the initial test migration:

   Click here to see screenshot of the settings

   

2. After taking the above screenshot, I changed the following in the "Who has access" and "Manage access" settings, so that non-leadership people do not see the beta project board since we are not currently using it yet and it was confusing people seeing two project boards:

   • Under "Who has access" section, I changed the "Base role" from Write to No access
   • Under the "Manage access", I removed website and website-write teams.

3. Read the info in the blue box about Migrating your project to Projects (beta): https://docs.github.com/en/issues/trying-out-the-new-projects-experience/migrating-your-project . Some highlights:

   • "If the archive limit of 10,000 items is reached, additional items will not be migrated."
   • "Automation will not be migrated."
   • "Triage, archive, and activity will not be migrated."

[ExperimentsInHonesty]

It seems like when we migrated, it brought the automation rules at the time. When we tested this by changing what happens to reopened PRs (no longer added to in progress, instead added to PR needs review) it did not update the automations listed here: https://github.com/orgs/hackforla/projects/15/workflows/1671601 Although it looks like not all of the automations got migrated, because reopening an issue not listed there.

[JessicaLucindaCheng]

Copying notes related to the board migration from the Dev/PM agenda on 2022-09-19 (#2607 (comment)) to here:

• Board migration
  • PM dislikes how the top nav/menu is for the beta project board (but this isn't something we at HfLA can do anything about.)
  • The automations dealing with issues within certain columns should still work. However, the columns are now called "Status" in the beta projects.
  • PM dislikes how the beta projects are at the org level and can't be created at the repo level (but this isn't something we at HfLA can do anything about).

[JessicaLucindaCheng]

From Slack on 2022-09-24, copying Bonnie's Slack post on why we should hold off on migrating to the beta project board to here:

The website team had a 2 hour meeting with GitHub and here is why we want everyone to hold off

1. When you make the board it will have to have your project name, unlike the current boards, because they will all be in one place, so having multiple project boards be called "project board" or project management" will make it impossible to know which one is yours.
   a. Please add your project initials to your project board now. (e.g., Food oasis project board would be called FOLA: Project Management)
2. Currently, they will be changing this hopefully soon, when you go to one of the new boards, you leave the menu for your project repo. So moving from board to issues to wiki tabs is impossible. You have to navigate back to your repo and the menu of how to do that has changed to a new placement that is not intuitive.
3. Many projects have links in their wiki to the current boards. Each project will need to write up a migration plan for what links will need to be changed to make sure there are not out of date links on their project
4. Many projects have cards on their project board. These will be automatically converted to draft issues and they don't work the same way as cards, so that information will need to be part of the projects migration plan. Probably these links will need to be moved to the wiki.

Additional post by Bonnie:

See also this wiki page with details
https://github.com/hackforla/admin/wiki/GitHub:-Project-Boards

[ExperimentsInHonesty]

This issue needs to be re written or replaced

The following tests need to be performed by someone who has a classic project board on their forked copy of the repo

Does a github action that moves an issue from QA on close to the done column (designed for the old board to move an issue to another column) still work on the new board

• figure out what GHA moves the issue with label refactor to the done column from the QA after its closed.
• install the GHA
• add two test issues (one with the label and without)
• add issues to the project board (on your repo)
• migrate the project board (on your repo)
• close both the issues
• see if it worked

Does the GitHub Action need to be changed for the new project board, or does it work there automatically. Does it still work on the old board.

• after you do the above
• reopen project board classic that just was migrated (look in projects classic, closed)
• reopen the issues and move them to in progress column/status on both boards
• close both the issues
• see if it worked on both boards

Create a new issue and see which board it gets added to

Do we have to revise the GHA that adds it, so it goes to both?

https://github.com/hackforla/website/blob/gh-pages/.github/workflows/move-closed-issues.yaml
https://github.com/hackforla/website/blob/gh-pages/github-actions/move-closed-issues/sort-closed-issues.js

[t-will-gillis]

Testing in my repo:

Does a github action that moves an issue from QA on close to the done column (designed for the old board to move an issue to another column) still work on the new board

• figure out what GHA moves the issue with label refactor to the done column from the QA after its closed.
• install the GHA
• add two test issues (one with the label and without)
• add issues to the project board (on your repo)
• migrate the project board (on your repo)
• close both the issues
• see if it worked --> Didn't work

Does the GitHub Action need to be changed for the new project board, or does it work there automatically. Does it still work on the old board.

• after you do the above
• reopen project board classic that just was migrated (look in projects classic, closed)
• reopen the issues and move them to in progress column/status on both boards
• close both the issues
• see if it worked on both boards --> Did not always work see below
• Create a new issue and see which board it gets added to --> see below
• Do we have to revise the GHA that adds it, so it goes to both?

https://github.com/hackforla/website/blob/gh-pages/.github/workflows/move-closed-issues.yaml
https://github.com/hackforla/website/blob/gh-pages/github-actions/move-closed-issues/sort-closed-issues.js

[t-will-gillis]

@ExperimentsInHonesty

3 issues opened, and 3 corresponding PRs generated before Migration:

• 1 role: back end/devOps,
• 1 role: front end,
• 1 feature: refactor GHAs

After migration, I reviewed, approved, merged all 3 PRs:

• Failure: pull-request-trigger.yml at step "Gather-Merged-PRs- had failure with "alex-page" automation: could not find 'Project Board' or column name, so automation to remove the PR did not work.
• Failure: after the PR was merged, move-closed-issue.yml failed at the step "run-alex-page-automation": could not find "Project Board" or "QA" column
• all three issues sent to "QA" column after PR was merged and issue closed

Then I reopened the "Project Board(classic)" and also reopened the 3 issues, placed each issue in "In Progress" for both boards, then closed the issues again:

• on all 3, move-closed-issues.yml ran without errors,
• issue with role: back end/devOps: was sorted to: "Done" on (classic), --> However, went to "QA" on new board
• issue with role: front end sorted to "QA" on both boards
• issue with feature: refactor GHAs sorted to "QA" both boards
• Side note: the "Project Board (classic)" does not quite function the same way after reopening, for example it does not let me rearrange cards or more cards to a different column by dragging. Also, for some odd reason an additional new-not-classic Project Board has appeared that includes columns that I had created and deleted a long time ago as well as long deleted issues.

Opened a new issue with both boards active. The new issue only opened in "Project Board (classic)"

Re-closed the classic board and opened a new issue.

• Failure: Issue does not go to the board, issue-trigger.yml fails at "alex-page"

So far, the automations that involve moving issues to specific columns are the ones not working correctly and/or are failing. These include automations that use alex-page/github-project-automation-plus@v0.9.0 (three workflows):

• move-closed-issues.yml,

• pull-request-trigger.yml, and

• issue-trigger.yml

• Also includes schedule-fri-0700.yml, which although it runs without error, it does not actually do anything. It is supposed to look for issues in the column matching the "IN_PROGRESS_COLUMN_ID" but it does not look for or find the column, and then apparently finishes without doing anything.

[t-will-gillis]

@ExperimentsInHonesty

Updates: in the new Project Board / Project Board Beta's "Default workflows", there are options for adding new issues and new PRs to the Board- This makes much of the refactoring easier.

Created issues:

• Prep for Project Board Migration: GitHub Actions / Workflows Part 1 of 2  #6993
• Post Project Board Migration: REST API => GraphQL Part 2 of 2 #6994

This note from the 'Sunset Notice' seems to imply that GitHub is retiring the REST APIs in November- which is reason for the second issue (6994). Even if this isn't the case, it might still be a good idea to start planning to convert to GraphQL anyway...

[t-will-gillis]

Affected workflows

Found so far

Issue Workflow	Step	Problem	If It Isn't Fixed
#6996 issue-trigger.yml	"Add-Issue-To-Project-Board"	OK if code removed and we use PBB "Default workflows" #6995	n/a
" "	"Ask-For-Preliminary-Update"	.js file references 'column'	won't post the Prelim. Update comments (option: we could revert #6811 until fixed)
#6997 pull-request-trigger.yml	"Move-New-PRs-To-Project-Board"	OK if code removed and we use PBB "Default workflows" #6995	n/a
" "	"Gather-Merged-PRs" etc.	alex-page action + 'column' Note: closed PRs, Issues can be Auto-Archived in PBB #6995	Merged PRs, Closed/unmerged PRs won't be removed from Project Board
#6998 schedule-fri-0700.yml	"Add-Update-Label-Weekly" add-label.js	The .js file refs 'column'	Update labels eg. 2 weeks inactive, To update! won't be added to issues
#7039 move-closed-issues.yaml	"move-closed-issue" sort-closed-issues.js	alex-page action + 'column' Note: closed issues sorted to "QA" in PBB #6995	All closed issues will sort to 'QA' status

[ExperimentsInHonesty]

• Submit a DRAFT PR to Remove the following workflows

• issue-trigger.yml

• pull-request-trigger.yml

• wr-pull-request-trigger.yml

• Submit a DRAFT PR to Revert Added logic to an Issue's preliminary comment with accompanying messages #6811

[ExperimentsInHonesty]

• Rewrite an issue to remove the reference to columns in the workflow, so that it adds the labels in any column.

• Project Board Migration: refactoring of "Schedule Friday 0700" workflow  #6998
  schedule-fri-0700.yml
  "Add-Update-Label-Weekly"add-label.js
  The .js file refs 'column'
  labels eg. 2 weeks inactive,To update! 

• Make a new issue (put in icebox) to refactor schedule-fri-0700.yml after the migration so that it goes back to only applying to the required columns

[t-will-gillis]

Additional notes:
We need to revise the wiki docs including:

• revise The "Hack for LA's GitHub Actions" Wiki guide needs additional explanation #6537 (comment)

[ExperimentsInHonesty]

Bonnie to review all open comments and see if this issue can be closed.