Maintenance: labels strategy & automation for Issues and PRs

[saragerion]

Description of the feature request

Minor feature but useful for governance of external contributions by the AWS community.
It would be good to review labels used for Github Issues and PR's.

• Review the existing labels and add, edit, delete labels also based on other runtimes
• When applicable, automate the assignment of labels based on specific criteria
• Nice to have: make sure that there is consistency and labels convention (uppercase vs lower case, spaces, characters used, ...)

Problem statement

Inspired by this PR: aws-powertools/powertools-lambda-python#894
Without proper labelling the runtime maintainers and the AWS community can be confused about the current status of the item (issue or PR).

Summary of the feature

N/A

Code examples

See other runtimes:
https://github.com/awslabs/aws-lambda-powertools-python/labels
https://github.com/awslabs/aws-lambda-powertools-java/labels

Benefits for you and the wider AWS community

Improved transparency and governance.

Describe alternatives you've considered

N/A

Additional context

N/A

Related issues, RFCs

N/A

[dreamorosi]

Agree, additionally it'd be great if all issues could go by default in the "Issues" project as "Needs clarification or refinement (untriaged)" as well.

[dreamorosi]

Related to #133

As discussed with @ijemmy, I'd be happy to support with this one.

Here is my proposal how we could structure the labels for issues and pull requests. Please let me know if you'd like to change something!

Labels

General

Label	Applies to	Comment
good-first-issue	Issues	Something that is suitable for those who want to start contributing
help-wanted	Issues	We would really appreciate some support from community for this one
breaking-change	Issues, PRs	Introduces a breaking change
duplicate	Issues	This issue is a duplicate of an existing one; a reference is required in the comments

Scope

Applies to: Issues, PRs

Label	Comment
scope/packages	Affects one or multiple tools (packages)
scope/docs	Affects documentation, README, CONTRIBUTING, and other similar docs
scope/tests	Affects tests only; applies when there are no changes in the tools (packages)
scope/infrastructure	Affects the repo infrastructure, including GitHub Actions, workflows, compiler settings, ...

Status

Applies to: Issues

These labels do not resemble the columns in the Issues project, but rather focus on the actual issue state. The label helps understand if any action is required by maintainers or by the author(s).

Label	Comment
status/submitted	A newly created issue, auto-assigned on creation
status/discussing	The issue needs to be discussed, elaborated, or refined; applies to both external and internal ones
status/confirmed	The scope is clear, ready for implementation
status/on-hold	There are some blockers that prevent this from being implemented (technical or any others); needs to be periodically revised
status/rejected	wontfix - this is something we will not be working on (at least, not in the measurable future); might be changed later though

Type

Identifies issues and PRs by their type. This label will be auto-assigned to all PRs linked to issues. For PRs without any issues (should be rarely the case), the label needs to be assigned manually.

Label	Applies to	Comment
type/rfc	Issues	Request for a new feature; to be discussed and confirmed
type/bug	Issues, PRs	Something isn't working
type/feature	Issues, PRs	A new feature linked to a type/rfc issue (e.g. partial implementation) or an enhancement for an existing feature
type/chore	Issues, PRs	Any non-functional change (docs, infrastructure, refactoring, ...)
type/dependency	Issues, PRs	Updates a dependency without any functional change (mostly dependabot's work)
type/question	Issues	Not really an issue, but a question posted as an issue; doesn't need to be implemented as a PR

Priority

Applies to: Issues, PRs

Identifies issues and PRs by their priority. Helps focus on things that really matter.
The PRs will get this label automatically from the issues they reference (if available).

Label	Comment
priority/high	Must Have
priority/medium	Should Have
priority/low	Could Have

Note: Won't Have is represented by status/rejected.

Source

Applies to: Issues, PRs

Identifies issues and PRs by their origin.
Helps keep track on the number of changes coming from the maintainers and from the community.

Label	Comment
source/internal	An issue or a PR created by maintainers
source/external	An issue or a PR created by the community or by customers

Packages

Applies to: Issues, PRs

Classifies the issues and the pull requests according to the changes needed in the tools (packages).
For issues, these labels can be assigned manually. For PRs, they will be assigned automatically based on the paths of the changed files. For issues and PRs not related to packages (like e.g. docs, infrastructure), no package/* labels will be assigned.
The label makes easier to identify the scope and see the statistics.

Label	Comment
package/common	Not always obvious for issues
package/logger
package/metrics
package/tracer

Pull request size

Applies to: PRs

Classifies the pull requests according to the number of changed code lines.
The label makes it easier to see how much time is needed for reviewing the PR.

Label	Number of changed code lines
size/XS	0-9
size/S	10-29
size/M	30-99
size/L	100-499
size/XL	500-999
size/XXL	1000+

Automation

Action	Automation for labels
New issue submitted	status/submitted and move to project Issues (1st column)
New issue submitted	source/internal or source/external based on submitter
Issue moved to 2nd column in the project Issues	remove status/submitted or status/discussing, add status/confirmed
Dependabot PR submitted	type/dependency
New PR submitted	package/* based on changed files
New PR submitted	size/* based on changes
New PR submitted	apply scope/*, priority/*, source/*, and type/* from referenced issue
Breaking change option selected in PR	breaking-change

[dreamorosi]

Thanks @AWSDB for the thorough proposal, I like it so far! Few notes/questions:

• Do you think it would make sense to add a scope/examples label, or would these fall under scope/docs?
• Would it be possible to try to label new issues with package/* & type/* based on the title? I know this won't work 100% of times but since we have templates in some cases it might work.
• What would happen to all the existing issues? Is there any way of re-labeling them according to the new system or should we do it manually (which is fine, just asking)

Happy to discuss the points you mentioned, @dreamorosi!

• scope/examples: I think, we can decide what better suits our needs - scope/examples is more descriptive, but at the same time it adds a new label that doesn't much differ from scope/docs. Examples could be also seen as documentation, but they are of course somewhat different from the actual docs published. What would you prefer - a dedicated category or fewer labels?
• Auto-labeling: While we could define some regex guesses or rules, I doubt we can make this reliable enough. That means, we will still need to inspect and review the assigned labels for every issue. Should we maybe start without that first, settle the new labeling schema, and then try to add auto-labeling for package/* and type/*?
• Label migration: Some of the labels can be just replaced (e.g. bug with type/bug) - so old issues and PRs will automatically get the new labels instead of old ones. Not needed labels can be simply deleted. A script (manual workflow) can be created for automatically assigning new labels where possible (like size/* or package/*). Some labels will need to be assigned manually though.

[saragerion]

Hi @AWSDB,
first of all thanks a lot for picking up this issue and detailing your proposal.
I noticed that the labels syntax has changes from key:value to key/value. I don't have a strong opinion about it, but do you find it more readable?

Scope

Are the possible values mutually exclusive? I am assuming not.
For PRs, it would be nice to add labels automatically based on changes happened on specific folders (docs, examples, .github). I would also add scope/cicd

Status

Looking at ways to reduce the manual tasks for maintainers, I think status/discussing is redundant. We acknowledge every open issue that is submitted, and then we politely reject it or agree on it.
Also, I am more keen on something like status/wontdo or similar rather than "status/rejected" as the latter feels more "confrontational" (in my opinion).

Type

It seems like this is a mix between our issue types and our semantic types.
Which one of the 2?

Pull request size

That's an interesting unit of measure. I normally look at PRs in terms of files.

[dreamorosi]

Agree with @saragerion here, I think automation is key to this topic. This way we can create a mechanism, on which we can iterate to improve it overtime, instead of relying on good intentions.

Not saying it has to be part of a single PR, I'd be happy to have one to set up labels and later automation but I'd like to at least get the ball rolling on this area before considering the issue as completed.

Thank you for the feedback @saragerion! Let me answer your questions.

key:value to key/value

It's rather category/value than key:value. E.g. compare with branch names like feature/something or bugfix/something. But we can surely use a colon instead of a forward slash keeping the key-value paradigm in mind - it's actually a matter of taste.

Scope
Are the possible values mutually exclusive?

I'd propose to keep them mutually exclusive (also for brevity). The scope label would describe the purpose, the main focus of the PR or issue. A packages PR can definitely contain tests (and actually should), as well as docs, but - from my point of view - labeling such PRs with all three labels (packages + docs + tests) adds not so much value. So I'd see the scope/* label not as "what's inside", but rather "what it's for".

Automated labeling for PRs is of course possible here (based on the changed content). scope/cicd fits into scope/infrastructure from my point of view. What do you think?

status/discussing

It's probably not the best naming. A better idea? My thoughts were: how would you tag issues that need some further input or clarification from the requester? "Feedback needed" or "On hold until clarified" etc. Do we want to distinguish such issues from the other ones? Otherwise, this label is redundant of course.

a mix between our issue types and our semantic types

Yes, it's a combination of both (a bit "condensed"). Do you feel like separating them would be beneficial? I tried to figure out the minimal number of types without being too verbose.

---

@dreamorosi, I agree - automation is key here. And I also believe some basic automation should be implemented right away, in the same PR - at least, the basic rules. This shouldn't bee too expensive to set up.