Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(rfcs): add triage automation rfc #24817

Merged
merged 4 commits into from
Nov 8, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions rfcs/shared/build-system/triage-automation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
# RFC: Triage automation (Shield)

<!--
An RFC can be anything. A question, a suggestion, a plan. The purpose of this template is to give some structure to help folks write successful RFCs. However, don't feel constrained by this template; use your best judgement.

Tips for writing a successful RFC:

- Simple plain words that make your point, fancy words obfuscate
- Try to stay concise, but don't gloss over important details
- Try to write a neutral problem statement, not one that motivates your desired solution
- Remember, "Writing is thinking". It's natural to realize new ideas while writing your proposal
-->

_contributors:_ _@hotell_

---

## Summary

We wanna make initial triage-ing issues a fully automated process.

## Problem statement

At the moment processing newly created issues is mostly manual work done by 2 people on shield duty. While this was a good start it wont scale as the codebase/team/contributors list grows. Thus we need to introduce automation to establish best possible contributors DX while they need our help.

<!--
Why are we making this change? What problem are we solving? What do we expect to gain from this?

This section is important as the motivation or problem statement is indepenent from the proposed change. Even if this RFC is not accepted this Motivation can be used for alternative solutions.

In the end, please make sure to present a neutral Problem statement, rather than one that motivates a particular solution
-->

## Detailed Design or Proposal

<!-- This is the bulk of the RFC. Explain the proposal or design in enough detail for the inteded audience to understand. -->

After we [introduced new github issues beta](https://github.com/microsoft/fluentui/pull/22337) for reporting issues and feature request, the experience for both contributors and shield devs was significantly improved. A followup that was discussed is to implement further automation by leveraging Github Actions, This document will describe the approach in better detail.

We would like to automate following steps:

1. project label assignment
2. package label assignment
3. assign issue to project board/team

**Pre-requirements:**

- normalization/unification of labels within monorepo for projects

_Current state:_

- 2 labels for v0
- https://github.com/microsoft/fluentui/labels?q=northstar
- 2 labels for v9
- https://github.com/microsoft/fluentui/labels/Fluent%20UI%20react-components%20%28v9%29
- https://github.com/microsoft/fluentui/labels/Fluent%20UI%20vNext
- per component labels
- same labels mixed for various fluent version https://github.com/microsoft/fluentui/labels/Component%3A%20Dialog

_Expected state:_

- 1 label for v0
- 1 label for v9
- label per published package
- "component labels" can stay as are but should be used solely for v8 (internal react package "sub-packages" https://github.com/microsoft/fluentui/labels/Component%3A%20Dialog)

### 1. project label assignment

> Note:
>
> - by "project" we mean one of libraries that exist within monorepo (v0,v8,v9,web-components)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this make sense with respect to the current expectations for partners ? We actively encourage partners to use @fluentui/react-components. Using package labels instead of component labels seems like we are fighting against our own recommendation to use the suite package

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure I understand the concern?

- by "project" we mean one of libraries that exist within monorepo (v0,v8,v9,web-components)

we do this today - add label based on project , NOT granular package.

probably you're asking about option 2 (2. package label assignment)?

can you pls clarify? ty

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I 'm asking about packge label assignment why should we oblige users to select the correct package in our monorepo when the only exposure they have (and we do it intentionally) is the library package ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

answered with proposal #24817 (comment)

Copy link
Contributor Author

@Hotell Hotell Oct 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated RFC d9b832f


**Workflow:**

Based on selected project within issue template, we should add appropriate label.

<img width="1108" alt="image" src="https://user-images.githubusercontent.com/1223799/190106473-fd007d87-f09a-4493-aafd-781a8404ae45.png">

> NOTE:
>
> - this is already implement via [github action](https://github.com/microsoft/fluentui/pull/24911)

### 2. package label assignment

> Note:
>
> - by "package" we mean official package name used to publish to npm registry without `@scope` prefix
> - by "Component name" we mean official package name transformed via following `toPascalCase(packageName.replace('@fluentui/react-'))`

**Workflow:**

Based on selected project within issue template, additional select should be dynamically generated so contributor can pick proper package.

<img width="569" alt="example of flow" src="https://user-images.githubusercontent.com/1223799/190431945-56fcd343-6eb7-4145-b2b2-62f5919b8c31.png">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @ling1726 partners are supposed to use just the suite package, we should not ask them to guess the individual package.
Can we change this to list of components instead of packages?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to list of components instead of packages?

We can, but it might be impossible to identify those as packages export various Controls.

Two approaches come to my mind:

  1. use storybooks root chapters for those
  • image
  • which might be quite cumbersome to implement and stay in sync.
  1. Pascal Casing package names without scope
  • robust and easy to sync
  • Example: @fluentui/react-text -> Text , @fluentui/react-menu-> Menu

I'd recommend to go with 2), WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

option 2 good enough 👍


Unfortunately **[dynamic input controls are not possible with github issues beta](https://github.com/community/community/discussions/29067)** which leaves us with following options how to implement this:

Comment on lines +92 to +97
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this allow you to select multiple packages? Or in case the issue affected multiple packages, how would the scenario proceed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question, we can enable multiple items selected with github forms https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema#dropdown.

What would be the use-case ?

to provide more context:

Our template is aimed mainly for 3rd party with workflow.

Meaning: "Im using fluent, Im using this component and it doesn't work for me."

From my experience user reports issue for the immediate thing that comes in his way( component A ).

Additionally one can dig deeper and find that the issue is within some underlying component (Component B) - thus the reporter should pick that one.

In general having fine grained issue reports reduces the complexity and time to identify the problem. This is omnipresent everywhere - dissecting task to smaller ones/ atomic commits/small PR's.

to wrap up I'm IMO it shouldn't be needed. but as mentioned the option is there

1. use only package names or Component name in "Library" picker

- cons
- huge list because we have quite a lot packages
- pros
- only 1 issue/feature template that we need to maintain
- removes need of `1. project label assignment` as labeling would be handled in one step

2. creating feature/bug templates per library

- cons
- 2 (feature/bug) x N templates (N is number of libraries in monorepo)
- pros
- clear separation that can be tweaked to particular library needs if needed
Comment on lines +106 to +111
Copy link
Collaborator

@JustSlone JustSlone Oct 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 vote for option 2

This breaks the decision up for contributors into phases

  1. Decide the library choose a template (should be easy)
  2. As part of filing choose a component from a smaller list (easier or the same as other options)

I also think that the likelihood of having differences between library template questions is high, so having separate templates for each will have other benefits.


3. creating completely custom issue form solution (example [vuejs](https://github.com/vuejs/vue/issues/new/choose))

- cons
- big effort in terms of development and maintenance
- pros
- complete control about issue reporting
- another possible dog food for v9 controls

**Workflow approach - summary:**

After feedback and additional discussion the majority is in favor of "2. creating feature/bug templates per library", where the select list will contain "Component names".

Simplified example of select box option for project issue template:

```
Component:
- Text
- Menu
- Dialog
- ...
```

Based on this, all project labels should adhere to unified pattern based on issue origin(project):

1. issue origin: v9 template:

```
Text -> package: react-text, Fluent UI react-components (v9)
Menu -> package: react-menu, Fluent UI react-components (v9)
Dialog -> package: react-dialog, Fluent UI react-components (v9)
```

2. issue origin: v8 template:

```
Text -> package: react, Fluent UI react (v8)
Menu -> package: react, Fluent UI react (v8)
Dialog -> package: react, Fluent UI react (v8)
Chart -> package: react-charting, Fluent UI react (v8)
```

3. issue origin: v0 template:

```
Text -> package: react-northstar, Fluent UI react-northstar (v0)
Menu -> package: react-northstar, Fluent UI react-northstar (v0)
Dialog -> package: react-northstar, Fluent UI react-northstar (v0)
```

4. issue origin: web-components template:

```
Text -> package: web-components
Menu -> package: web-components
```
Comment on lines +154 to +167
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For cases 3. and 4. above, will we still have a label indicating the component? These have been pretty helpful for v8.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


**Q&A:**

**1. How will we maintain list of packages in issue template:**

On library creation/removal we will tweak our issue/feature template automatically.

> Note:
>
> - this can also happen only after created package has been published to npm registry ( post release GH action)

### 3. assign issue to project board/team

Based on assigned labels, we can automate assignment of issues based on our source of truth of ownership which is `CODEOWNERS` file.

> 💡 NOTE:
>
> - every issue will still contain "Triage needed" flag, so it can be verified by person on shield duty

**Workflow:**

```mermaid
graph TD;
I("issue template created")-->L("labels added")-->C("parse codeowners");
subgraph "parse codeowners"
C-->D("find owner based on package")
end
D-->A("add issue to unified board")-->AA("pick team based on ownership definition");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The role of the shield is to:

  1. Verify the issue is actionable.
  2. Set the priority for the issue.

In order to be able to do that, shield often needs to talk back to the reporter and/or debug the issue. As a side effect, the person on shield widen their knowledge to cover the whole library (useful for both support and development).

If the bot assigns the issues automatically, then:

  1. We are siloing crews.
  2. We require each crew to do their own "shield" (not necessarily having a dedicated person) to cover the two points above on a daily basis.

For that reason I would prefer the bot to suggest the owner but would still let shield to the check/assignment.

Copy link
Contributor Author

@Hotell Hotell Sep 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For that reason I would prefer the bot to suggest the owner but would still let shield to the check/assignment.

that's fine, it will still contain "needs triage" label, so person on shield can double check. we can create a view in project board that will show all "need triage" items for better visibility and post-bot processing.

WDYT ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me. I still think that shield is valuable and should visit every incoming issue for triage/investigation.
What you are proposing is achieving that while reducing manual repetitive work = win-win

```

## Pros and Cons

**Cons:**

- some amount of maintenance

**Pros:**

- improved issues reporting experience
- improved post-triaging experience and prioritization
- no need for:
- human interaction for initial triaging
- manual communication that can "get lost" and can take extra time (time-zone constraints)
- extra number of humans needed on Shield

## Discarded Solutions

<!-- As you enumerate possible solutions, try to keep track of the discarded ones. This should include why we discarded the solution. -->

NONE

## Open Issues

Not applicable

<!-- Optional section, but useful for first drafts. Use this section to track open issues on unanswered questions regarding the design or proposal. -->