-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Onboard the GitHub Action based Issue-Labeler #78426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
arunchndr
merged 3 commits into
dotnet:main
from
jeffhandley:jeffhandley/issue-labeler-v2.0.0
May 8, 2025
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # Workflow template imported and updated from: | ||
| # https://github.com/dotnet/issue-labeler/wiki/Onboarding | ||
| # | ||
| # See labeler.md for more information | ||
| # | ||
| # Regularly restore the prediction models from cache to prevent cache eviction | ||
| name: "Labeler: Cache Retention" | ||
|
|
||
| # For more information about GitHub's action cache limits and eviction policy, see: | ||
| # https://docs.github.com/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: "24 19 * * *" # 19:24 every day (arbitrary time daily) | ||
|
|
||
| workflow_dispatch: | ||
| inputs: | ||
| cache_key: | ||
| description: "The cache key suffix to use for restoring the model from cache. Defaults to 'ACTIVE'." | ||
| required: true | ||
| default: "ACTIVE" | ||
|
|
||
| env: | ||
| CACHE_KEY: ${{ inputs.cache_key || 'ACTIVE' }} | ||
|
|
||
| jobs: | ||
| restore-cache: | ||
| # Do not automatically run the workflow on forks outside the 'dotnet' org | ||
| if: ${{ github.event_name == 'workflow_dispatch' || github.repository_owner == 'dotnet' }} | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| type: ["issues", "pulls"] | ||
| steps: | ||
| - uses: dotnet/issue-labeler/restore@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: ${{ matrix.type }} | ||
| cache_key: ${{ env.CACHE_KEY }} | ||
| fail-on-cache-miss: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| # Workflow template imported and updated from: | ||
| # https://github.com/dotnet/issue-labeler/wiki/Onboarding | ||
| # | ||
| # See labeler.md for more information | ||
| # | ||
| # Predict labels for Issues using a trained model | ||
| name: "Labeler: Predict (Issues)" | ||
|
|
||
| on: | ||
| # Only automatically predict area labels when issues are first opened | ||
| issues: | ||
| types: opened | ||
|
|
||
| # Allow dispatching the workflow via the Actions UI, specifying ranges of numbers | ||
| workflow_dispatch: | ||
| inputs: | ||
| issues: | ||
| description: "Issue Numbers (comma-separated list of ranges)." | ||
| required: true | ||
| cache_key: | ||
| description: "The cache key suffix to use for restoring the model. Defaults to 'ACTIVE'." | ||
| required: true | ||
| default: "ACTIVE" | ||
|
|
||
| env: | ||
| # Do not allow failure for jobs triggered automatically (as this causes red noise on the workflows list) | ||
| ALLOW_FAILURE: ${{ github.event_name == 'workflow_dispatch' }} | ||
|
|
||
| LABEL_PREFIX: "Area-" | ||
| THRESHOLD: 0.40 | ||
|
|
||
| jobs: | ||
| predict-issue-label: | ||
| # Do not automatically run the workflow on forks outside the 'dotnet' org | ||
| if: ${{ github.event_name == 'workflow_dispatch' || github.repository_owner == 'dotnet' }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| issues: write | ||
| steps: | ||
| - name: "Restore issues model from cache" | ||
| id: restore-model | ||
| uses: dotnet/issue-labeler/restore@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: issues | ||
| fail-on-cache-miss: ${{ env.ALLOW_FAILURE }} | ||
| quiet: true | ||
|
|
||
| - name: "Predict issue labels" | ||
| id: prediction | ||
| if: ${{ steps.restore-model.outputs.cache-hit == 'true' }} | ||
| uses: dotnet/issue-labeler/predict@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| issues: ${{ inputs.issues || github.event.issue.number }} | ||
| label_prefix: ${{ env.LABEL_PREFIX }} | ||
| threshold: ${{ env.THRESHOLD }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| continue-on-error: ${{ !env.ALLOW_FAILURE }} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # Workflow template imported and updated from: | ||
| # https://github.com/dotnet/issue-labeler/wiki/Onboarding | ||
| # | ||
| # See labeler.md for more information | ||
| # | ||
| # Predict labels for Pull Requests using a trained model | ||
| name: "Labeler: Predict (Pulls)" | ||
|
|
||
| on: | ||
| # Per to the following documentation: | ||
| # https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_request_target | ||
| # | ||
| # The `pull_request_target` event runs in the context of the base of the pull request, rather | ||
| # than in the context of the merge commit, as the `pull_request` event does. This prevents | ||
| # execution of unsafe code from the head of the pull request that could alter the repository | ||
| # or steal any secrets you use in your workflow. This event allows your workflow to do things | ||
| # like label or comment on pull requests from forks. | ||
| # | ||
| # Only automatically predict area labels when pull requests are first opened | ||
| pull_request_target: | ||
| types: opened | ||
|
|
||
| # Configure the branches that need to have PRs labeled | ||
| branches: | ||
| - main | ||
|
|
||
| # Allow dispatching the workflow via the Actions UI, specifying ranges of numbers | ||
| workflow_dispatch: | ||
| inputs: | ||
| pulls: | ||
| description: "Pull Request Numbers (comma-separated list of ranges)." | ||
| required: true | ||
| cache_key: | ||
| description: "The cache key suffix to use for restoring the model. Defaults to 'ACTIVE'." | ||
| required: true | ||
| default: "ACTIVE" | ||
|
|
||
| env: | ||
| # Do not allow failure for jobs triggered automatically (this can block PR merge) | ||
| ALLOW_FAILURE: ${{ github.event_name == 'workflow_dispatch' }} | ||
|
|
||
| LABEL_PREFIX: "Area-" | ||
| THRESHOLD: 0.40 | ||
|
|
||
| jobs: | ||
| predict-pull-label: | ||
| # Do not automatically run the workflow on forks outside the 'dotnet' org | ||
| if: ${{ github.event_name == 'workflow_dispatch' || github.repository_owner == 'dotnet' }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| pull-requests: write | ||
| steps: | ||
| - name: "Restore pulls model from cache" | ||
| id: restore-model | ||
| uses: dotnet/issue-labeler/restore@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: pulls | ||
| fail-on-cache-miss: ${{ env.ALLOW_FAILURE }} | ||
| quiet: true | ||
|
|
||
| - name: "Predict pull labels" | ||
| id: prediction | ||
| if: ${{ steps.restore-model.outputs.cache-hit == 'true' }} | ||
| uses: dotnet/issue-labeler/predict@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| pulls: ${{ inputs.pulls || github.event.number }} | ||
| label_prefix: ${{ env.LABEL_PREFIX }} | ||
| threshold: ${{ env.THRESHOLD }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| continue-on-error: ${{ !env.ALLOW_FAILURE }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # Workflow template imported and updated from: | ||
| # https://github.com/dotnet/issue-labeler/wiki/Onboarding | ||
| # | ||
| # See labeler.md for more information | ||
| # | ||
| # Promote a model from staging to 'ACTIVE', backing up the currently 'ACTIVE' model | ||
| name: "Labeler: Promotion" | ||
|
|
||
| on: | ||
| # Dispatched via the Actions UI, promotes the staged models from | ||
| # a staged slot into the prediction environment | ||
| workflow_dispatch: | ||
| inputs: | ||
| issues: | ||
| description: "Issues: Promote Model" | ||
| type: boolean | ||
| required: true | ||
| pulls: | ||
| description: "Pulls: Promote Model" | ||
| type: boolean | ||
| required: true | ||
| staged_key: | ||
| description: "The cache key suffix to use for promoting a staged model to 'ACTIVE'. Defaults to 'staged'." | ||
| required: true | ||
| default: "staged" | ||
| backup_key: | ||
| description: "The cache key suffix to use for backing up the currently active model. Defaults to 'backup'." | ||
| default: "backup" | ||
|
|
||
| permissions: | ||
| actions: write | ||
|
|
||
| jobs: | ||
| promote-issues: | ||
| if: ${{ inputs.issues }} | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: "Promote Model for Issues" | ||
| uses: dotnet/issue-labeler/promote@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "issues" | ||
| staged_key: ${{ inputs.staged_key }} | ||
| backup_key: ${{ inputs.backup_key }} | ||
|
|
||
| promote-pulls: | ||
| if: ${{ inputs.pulls }} | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: "Promote Model for Pull Requests" | ||
| uses: dotnet/issue-labeler/promote@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "pulls" | ||
| staged_key: ${{ inputs.staged_key }} | ||
| backup_key: ${{ inputs.backup_key }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| # Workflow template imported and updated from: | ||
| # https://github.com/dotnet/issue-labeler/wiki/Onboarding | ||
| # | ||
| # See labeler.md for more information | ||
| # | ||
| # Train the Issues and Pull Requests models for label prediction | ||
| name: "Labeler: Training" | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| type: | ||
| description: "Issues or Pull Requests" | ||
| type: choice | ||
| required: true | ||
| default: "Both" | ||
| options: | ||
| - "Both" | ||
| - "Issues" | ||
| - "Pull Requests" | ||
|
|
||
| steps: | ||
| description: "Training Steps" | ||
| type: choice | ||
| required: true | ||
| default: "All" | ||
| options: | ||
| - "All" | ||
| - "Download Data" | ||
| - "Train Model" | ||
| - "Test Model" | ||
|
|
||
| limit: | ||
| description: "Max number of items to download for training/testing the model (newest items are used). Defaults to the max number of pages times the page size." | ||
| type: number | ||
| page_size: | ||
| description: "Number of items per page in GitHub API requests. Defaults to 100 for issues, 25 for pull requests." | ||
| type: number | ||
| page_limit: | ||
| description: "Maximum number of pages to download for training/testing the model. Defaults to 1000 for issues, 4000 for pull requests." | ||
| type: number | ||
| cache_key_suffix: | ||
| description: "The cache key suffix to use for staged data/models (use 'ACTIVE' to bypass staging). Defaults to 'staged'." | ||
| required: true | ||
| default: "staged" | ||
|
|
||
| env: | ||
| CACHE_KEY: ${{ inputs.cache_key_suffix }} | ||
| REPOSITORY: ${{ github.repository }} | ||
| LABEL_PREFIX: "Area-" | ||
| THRESHOLD: "0.40" | ||
| LIMIT: ${{ inputs.limit }} | ||
| PAGE_SIZE: ${{ inputs.page_size }} | ||
| PAGE_LIMIT: ${{ inputs.page_limit }} | ||
|
|
||
| jobs: | ||
| download-issues: | ||
| if: ${{ contains(fromJSON('["Both", "Issues"]'), inputs.type) && contains(fromJSON('["All", "Download Data"]'), inputs.steps) }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| issues: read | ||
| steps: | ||
| - name: "Download Issues" | ||
| uses: dotnet/issue-labeler/download@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "issues" | ||
| cache_key: ${{ env.CACHE_KEY }} | ||
| repository: ${{ env.REPOSITORY }} | ||
| label_prefix: ${{ env.LABEL_PREFIX }} | ||
| limit: ${{ env.LIMIT }} | ||
| page_size: ${{ env.PAGE_SIZE }} | ||
| page_limit: ${{ env.PAGE_LIMIT }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
|
|
||
| download-pulls: | ||
| if: ${{ contains(fromJSON('["Both", "Pull Requests"]'), inputs.type) && contains(fromJSON('["All", "Download Data"]'), inputs.steps) }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| pull-requests: read | ||
| steps: | ||
| - name: "Download Pull Requests" | ||
| uses: dotnet/issue-labeler/download@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "pulls" | ||
| cache_key: ${{ env.CACHE_KEY }} | ||
| repository: ${{ env.REPOSITORY }} | ||
| label_prefix: ${{ env.LABEL_PREFIX }} | ||
| limit: ${{ env.LIMIT }} | ||
| page_size: ${{ env.PAGE_SIZE }} | ||
| page_limit: ${{ env.PAGE_LIMIT }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
|
|
||
| train-issues: | ||
| if: ${{ always() && contains(fromJSON('["Both", "Issues"]'), inputs.type) && contains(fromJSON('["All", "Train Model"]'), inputs.steps) && contains(fromJSON('["success", "skipped"]'), needs.download-issues.result) }} | ||
| runs-on: ubuntu-latest | ||
| permissions: {} | ||
| needs: download-issues | ||
| steps: | ||
| - name: "Train Model for Issues" | ||
| uses: dotnet/issue-labeler/train@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "issues" | ||
| data_cache_key: ${{ env.CACHE_KEY }} | ||
| model_cache_key: ${{ env.CACHE_KEY }} | ||
|
|
||
| train-pulls: | ||
| if: ${{ always() && contains(fromJSON('["Both", "Pull Requests"]'), inputs.type) && contains(fromJSON('["All", "Train Model"]'), inputs.steps) && contains(fromJSON('["success", "skipped"]'), needs.download-pulls.result) }} | ||
| runs-on: ubuntu-latest | ||
| permissions: {} | ||
| needs: download-pulls | ||
| steps: | ||
| - name: "Train Model for Pull Requests" | ||
| uses: dotnet/issue-labeler/train@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "pulls" | ||
| data_cache_key: ${{ env.CACHE_KEY }} | ||
| model_cache_key: ${{ env.CACHE_KEY }} | ||
|
|
||
| test-issues: | ||
| if: ${{ always() && contains(fromJSON('["Both", "Issues"]'), inputs.type) && contains(fromJSON('["All", "Test Model"]'), inputs.steps) && contains(fromJSON('["success", "skipped"]'), needs.train-issues.result) }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| issues: read | ||
| needs: train-issues | ||
| steps: | ||
| - name: "Test Model for Issues" | ||
| uses: dotnet/issue-labeler/test@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "issues" | ||
| cache_key: ${{ env.CACHE_KEY }} | ||
| repository: ${{ env.REPOSITORY }} | ||
| label_prefix: ${{ env.LABEL_PREFIX }} | ||
| threshold: ${{ env.THRESHOLD }} | ||
| limit: ${{ env.LIMIT }} | ||
| page_size: ${{ env.PAGE_SIZE }} | ||
| page_limit: ${{ env.PAGE_LIMIT }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
|
|
||
| test-pulls: | ||
| if: ${{ always() && contains(fromJSON('["Both", "Pull Requests"]'), inputs.type) && contains(fromJSON('["All", "Test Model"]'), inputs.steps) && contains(fromJSON('["success", "skipped"]'), needs.train-pulls.result) }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| pull-requests: read | ||
| needs: train-pulls | ||
| steps: | ||
| - name: "Test Model for Pull Requests" | ||
| uses: dotnet/issue-labeler/test@46125e85e6a568dc712f358c39f35317366f5eed # v2.0.0 | ||
| with: | ||
| type: "pulls" | ||
| cache_key: ${{ env.CACHE_KEY }} | ||
| repository: ${{ env.REPOSITORY }} | ||
| label_prefix: ${{ env.LABEL_PREFIX }} | ||
| threshold: ${{ env.THRESHOLD }} | ||
| limit: ${{ env.LIMIT }} | ||
| page_size: ${{ env.PAGE_SIZE }} | ||
| page_limit: ${{ env.PAGE_LIMIT }} | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fascinating!
does this use the previous training from the old labeler by any chance or do we start over?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You have to re-train but now it's all self-service within each repository, and no services to run, storage to manage, or anything else outside GitHub Actions. But the logic for building the new model is the same as the old implementation (but muuuuuuuccch faster and easier). When you run the 'Labeler: Training' job, the job summary will show the new model's accuracy based on test results where it re-predicts existing issue/pulls in the repository and compares the new prediction against the existing labels.