Ruleset in Evaluate enforcement status unexpectedly shows as Fail in Rule insights

[dalbertom]

Select Topic Area

Question

Body

Hi there, I hope this is the right place to ask a question about Rulesets (but I can ask elsewhere if needed).

I recently created a new branch Ruleset, named history-lint, set the Enforcement status to Evaluate and targeted it to a single repository, set the branch criteria to both Default and All branches and chose the branch protection of "Require status checks to pass" where I added a check named history-lint as well.

Then I created a workflow in that repository with a job id history-lint that invokes a composite action from another repository where the actual check is performed.

I see the output of the REST api shows the check ran:

$ gh api repos/$ORG/$REPO/commits/$BRANCH_NAME/check-runs | jq -r '.check_runs[].name'
history-lint

However, the Rule insight shows the rule did not pass

I also noticed the status state is pending and no statuses are present

$ gh api repos/$ORG/$REPO/commits/$BRANCH_NAME/status | jq -r '.state, .statuses'
pending
[]

After manually setting the status via

$ gh api repos/$ORG/$REPO/statuses/$COMMIT_ID -f state='success' -f context='history-lint'

I now see the commit status, but the rule insight still shows it as failed.

Is there something missing in the workflow to allow the ruleset to pass? Perhaps the name of the check is expected to match something else?

Thanks

[dalbertom]

Once I enabled the rule turns out it was working as expected, so this looks like the Evaluation mechanism was incorrectly flagging the ruleset as failed when it would have indeed succeeded.

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬