Push rule public beta!

[patrick-knight]

We want to hear from you on push rules!

Who put an EXE file in my iOS mobile App repo? Why can I push a 20MiB mp3 when this repo is all .MD docs files? All these mysteries will soon be answerable and controllable.

It's been a year since we showed y'all the future of protecting branches with the public beta of repository rules.

But we had one lingering thought this whole time. How do we let you to prohibit certain things from entering a repository?

The repository rule release only let you protect refs, meaning the commit is in the repository already and can be reachable. Often, that's fine because rules are great for preventing unintended changes or enforcing governance decisions. But what about risky files in a repo due to size, type, or location? Thats where push rules come in.

If you’ve had a chance to try out the repo rules that are now in public beta, we’re curious to know your thoughts and how we can make this feature work better for you!

🐛 Known issues

~~ REST and GraphQL API will be available soon. ~~

When push rules are in active or evaluate mode the usage of both the /git/trees and /git/blobs API endpoints are restricted. - Fix rolled out May
15th 2024

June 13th 2024 Update

REST and GraphQL endpoints are available for push rules

Added a delegated bypass flow to the beta to allow one time requests to skip push rules.

[archon810]

@patrick-knight Can these be used to enforce LFS on large files? Something like "if file extension is *.foo AND file size > 20MB AND file is not on LFS => deny"? From the screenshot, it looks like it may work, although it's ambiguous - if the file bar.foo is using LFS and the rule set is for *.foo and size > 20MB, will it be denied no matter its size or will it be allowed no matter its size?

Can a rule be overridden by an admin on a case by case basis to push an exception to the rule?

[archon810]

Just tested this on a test repo, and unfortunately it doesn't work the way I would like it to.

The ruleset:

The test when not using LFS correctly blocks this 25MB file due to extension and size.

But the test when using LFS still blocks it due to extension:

testrepo> git push
Uploading LFS objects: 100% (1/1), 24 MB | 3.0 MB/s, done.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 2 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 546 bytes | 42.00 KiB/s, done.
Total 4 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
remote: error: GH013: Repository rule violations found for refs/heads/master.
remote: Review all repository rules at http://github.com/BLA/testrepo/rules?ref=refs%2Fheads%2Fmaster
remote:
remote: - GITHUB PUSH PROTECTION
remote:   —————————————————————————————————————————
remote:     Resolve the following violations before pushing again
remote:
remote:     - Files cannot include restricted file extensions.
remote:       Found 1 violation:
remote:
remote:       testlargeapks/foo.apk
remote:
To github.com:BLA/testrepo.git
 ! [remote rejected] master -> master (push declined due to repository rule violations)
error: failed to push some refs to 'github.com:BLA/testrepo.git'

[patrick-knight]

We evaluate each rule independently, so pushes cannot contain any *.apk or any file over 5MB. and because the apk hit the rule it will get blocked regardless of LFS.

Can a rule be overridden by an admin on a case by case basis to push an exception to the rule?

Not today, but it sure is something we are thinking a lot about as these rules do introduce new points of friction.

[archon810]

We evaluate each rule independently, so pushes cannot contain any *.apk or any file over 5MB. and because the apk hit the rule it will get blocked regardless of LFS.

In this case, sadly this system is not powerful enough right now. Please consider more complex rules that would AND instead of OR.

[patrick-knight]

Thanks for the feedback!

[archon810]

UI-wise, this is confusing IMO:

I took it as you can provide a comma-separated list of extensions, like zip, rar, but the end result looked wrong, which made me re-evaluate the instructions and only then provide one extension per entry.

I think this UI can be clarified, and also it should block commas.

[patrick-knight]

Thanks, that's a good note.

[TWiStErRob]

Comma is a valid file name (and therefore extension) character, please don't block it outright.

[Brend-Smits]

Awesome addition!
It seems like the REST API has not been updated yet to reflect this new addition. When can we expect an update for this API so we can access and update the rule programmatically?
See documentation: https://docs.github.com/en/rest/repos/rules?apiVersion=2022-11-28#update-a-repository-ruleset

[patrick-knight]

Thanks.

That API is not yet ready, but should be available quite soon.

[ViacheslavKudinov]

But we got it in advance 😎
integrations/terraform-provider-github#2235

[patrick-knight]

The REST and GraphQL endpoints are available for push rules.

[bewuethr]

I would like to prevent certain files, such as .DS_Store, from being committed. If use a file path restriction for **/.DS_Store, it does indeed prevent adding these files, but it also prevents removing them, because the rule is

Prevent commits that include changes in specified file paths from being pushed to the commit graph.

Ideally, I would be able to say "do not allow adding matching files, but deleting is okay".

[patrick-knight]

Thanks for the suggestion. How common would the removal be for folks that don't have bypass? Would you see that as a daily occurrence, weekly?

[bewuethr]

Once per repository, really, just to clean up the instances where we already such a file committed.

[Haarolean]

Why is this availble only for private repos? As an open source maintainer, I'd like to be able to deny pushes with 10-mb binary files in PRs without having to manually inspect every part of one.

[archon810]

Oh, right, push rulesets specifically, not rulesets in general. Disregard me!

Hopefully, they'll bring them to public repos after the beta period once it hits GA.

[patrick-knight]

Great question.

The limitation for public repo is driven by the behavior with forks. As push rules are designed to protect a repo and it's fork there are some concerns we have about how this would be received in the open source community. So we opted away from it for now. The public beta is a good way to listen to the feedback if we need to change that stance.

[JesperDramsch]

I think this feature would be good for open-source repos as well.

It would help against security escalation and downstream attacks by locking down certain files and filetypes.

[balupton]

FWIW, I initially thought this was a great idea for my repos, then recognised that git ignore files already do what I want.

[TWiStErRob]

@balupton the big difference is that .gitignore is client-side. Anyone can override it with a simple git add -f .... These rules protect the branches on the server side in GitHub, regardless of who you are and where you're pushing from.

[TWiStErRob]

I wanted to give feedback on a branch ruleset, but the "Give feedback" button opens this page...

[patrick-knight]

😬 Whoops. Thats should only be on the new push rule beta page. We'll get that fixed up soon.

If you want to provide feedback on the existing branch and tag rules, feel free to jump into this discussion post.

[siMobin]

Why is this ruleset not for the organization's private repo?

[patrick-knight]

Hey @siMobin,

Push rules are available for private and internal repos for GitHub Team and across organizations for GitHub Enterprise.

[chrisxaustin]

Excellent new feature, thank you for doing this.

My first rule set was to restrict workflow changes, only allowing maintainer/admin roles in each repo to reduce the risk of malicious 3rd party actions being added accidentally. This is only a workaround though - I'd strongly prefer to have an org-level policy to only use the workflow that's in main. I don't like that someone can modify the workflow in a PR and have it applied before it has been reviewed/approved.

[TWiStErRob]

@chrisxaustin Are you aware of this GitHub feature? It might not do exactly what you're after, but it's actually a more powerful way of protection in my reading. If your workflows call just a single command or script from your clone on the runner, your repository is vulnerable to external code execution, because an actor can raise a PR with the script/build script changed and that will execute even without touching the workflow files.

[bewuethr]

We set up a rule to prevent pushing .DS_Store files, but then disabled it (set it to "evaluate) because it prevented also removing those files (see here).

We bumped into another issue with the same rule: even though set to evaluate, it resulted in workflows being unable to push commits to a third-party repository. We use fjogeleit/yaml-update-action to update a version in a YAML file. This step started failing with

HttpError: Operation is blocked due to enforced push rulesets.

Deleting the push rule, even though it was only in evaluation mode, made the issue go away. Blocking in eval-only mode seems to be a bug.

We use a PAT to check out the repo and push to it, for what it's worth, and the bot user for which the PAT was generated has admin permissions on the repo; this lets the pushes override a "must create PR first" branch protection rule.

[nicholasgriffintn]

We were also getting this with Renovate, I think that's probably the same issue, in case anyone comes along.

[technologik]

@patrick-knight +1, we found evaluate mode restricting pushes of blobs during our automation that uploads blobs during PR creation.

[technologik]

Following up on the known restrictions on trees and blobs (especially during evaluate mode), @patrick-knight do we have a timeline for when a fix will be provided for this?

Thanks!

[patrick-knight]

We have an agree upon plan internally, roughly looking at a couple weeks.

[patrick-knight]

We rolled out a fix to resolve this behavior yesterday. Can you re-enable the rules causing this and let me if you are still seeing the same error(s)? Thanks!

[rcmarques3]

Hi @patrick-knight,

Using rulesets (and branch protection policies before that) we always faced a few issues while enforcing commit signatures.
Unsigned commit objects can get pushed to the repository, and it's the ref update that rejected certain operations that would make them part of the branches covered by the rules.
This ref validation sometimes causes problems for us, in situations where you're merging say a previously existing support branch into main. Those commits usually were valid at the time they were pushed to that branch, but when entering main's history their signature is once again checked.
This can fail due to:

1. the signing GPG key was set to expire, the user has generated a new one but incorrectly removed the old one from his profile
2. some of those committers are no longer valid user accounts
3. those past commits were part of a migration / GitHub onboarding and signatures were bypassed by an admin at the time

Any of these situations will cause a (signed) merge commit on main to get blocked, because some of the baggage it's bringing along does not presently yield a valid signature.

This new pre-receive enforcement seems like a great opportunity to address this use case.
Any chance you could support enforcing Code Signatures in the push rulesets, so we could use it there instead of branch rulesets and thus avoiding altogether what I described?

And BTW looking at the existing branch rulesets, enforcing commit metadata (ex: message patterns) does seem to follow the same kind of reasoning. Not that I have an immediate need for it, but it does make sense to me that it could be enforceable in the push phase as well.

[patrick-knight]

Great question @rcmarques3.

There's a lot of threads to pull here, but at the moment we don't have plans to move those into push rules. There are scenarios for large monorepos where keeping some of these behaviors as branch level rules continues to make sense.

[rcmarques3]

Of course having this as an option in the push rules doesn't mean it should be removed from branch rules :) because different use cases do need different policies.
Just wanted to bring this to your attention since IIRC the branch rulesets did suffer some small adjustments to their behavior in beta regarding signatures, but never really fixed the problem for us.
Push rulesets seem like the perfect lifecycle spot to enforce this validation - and I did suggest something along those lines at the time, but no such hook existed back then.
I understand if there's no bandwidth to address everything, but do hope this manages to get implemented at some point. Thanks!

[zoezhangmattr]

hey there, this is awesome feature, i tried to enable one, and bypass github app, but it doesnt seem work for the bypass, it still get the
Error: Unhandled error: HttpError: Operation is blocked due to enforced push rulesets

[patrick-knight]

We do have a know issue with the usage of both the /git/trees and /git/blobs API endpoints and rules. Do you know if those are in use in your scenario?

[zoezhangmattr]

heya thanks for reply, yes, the github app is using these endpoints. is there any plan to support by passing the github app, we use it in quite a few scenarios. hopefully the new feature can support it as well?

[patrick-knight]

We rolled out a fix to resolve this behavior yesterday. Can you re-enable the rules causing this and let me if you are still seeing the same error(s)? Thanks!

[DrNeil]

hi, Push Rules are a great idea, thank you for getting them this far.
I work with repos where it would be more valuable to have a list of allowed file types, than a restricted list.
We can keep adding to the restricted list, however eventually it is going to be every file type other than the handful we want to support in the repos.
Please consider the inverse of restricted files, as a rule; allowed file types

[patrick-knight]

Thanks @DrNeil great suggestion. We're taking a look at data across the beta on how we best help with scenarios like this as well as "bundling" common file and path types for future itterations.

[bodgit]

We use a tool called release-please on pushes to master which creates releases/tags based on semantic commits. Part of this involves updating the current version in .release-please-manifest.json and we don't want regular users to edit this file so I thought this would be a good candidate for a push protection rule.

We run the workflow as a dedicated GitHub App so I created the push ruleset with this App in the bypass list. The workflow cannot push a change to this file if the rulset is either active or evaluating, only disabled and nothing ever appears in the rule insights. I've confirmed with the gh cli that the token is for the correct App account so this feels like a bug currently.

Happy to provide more information if needed.

[patrick-knight]

Hey @bodgit is there use of the trees or blobs endpoints in that tool? Based on the behavior of blocking on evaluate it seems like it might be the same behavior we are working on.

[bodgit]

I can't easily tell, but the error message I get is something like:

Error: release-please failed: Error adding to tree: <commit sha>

So based on the presence of "tree", perhaps that is the case?

[patrick-knight]

We rolled out a fix to resolve this behavior yesterday. Can you re-enable the rules causing this and let me if you are still seeing the same error(s)? Thanks!

[bodgit]

Sorry, Ive only just seen your reply.

I've just re-enabled the ruleset and tested and I can confirm it now seems to work as expected; the ruleset blocks writes unless they are authored by the release-please workflow via our GitHub App.

Thanks for the update!

[TBeijen]

Stumbled upon this (push ruleset) looking for a way to protect pipeline definitions.

At a first glance seems to work as expected. And solves a real problem in a simple way that is otherwise hard to solve. 👌
(Some thoughts: Intuitively I was aiming at solving this in the platform that accesses our resources, in my current case Azure Devops, but thinking it through: We also trust GitHub to protect our main branch, and by that the integrity of our systems. This is similar)

About this warning:

Push rulesets are applied to a repository and its downstream repositories. To review the downstream repositories that will inherit this ruleset, visit the forks page.

I get it (I think) but I can also imagine it complicating contributing by others.

One more thought: When having enterprise plan, to what extent will it be possible to impose certain rules in bulk to all repositories?

[patrick-knight]

I can also imagine it complicating contributing by others.

That drove our initial work to support just private and internal repos to limit the scope of impact there.

You can 100% apply push and branch/tag rules to all repos in an org when on GitHub enterprise today, with the exception of Enterprise Server where push rules are not currently availible.

[oed-sassmannc]

Thanks for this feature! As far as we could test, it perfectly fits our needs to protect our Workflows.
Are there any notes about GA release? Can't wait to use this in production.

[patrick-knight]

We are targeting a GA sometime between June and August assuming everything goes to plan.

[DenisDiachkov]

Why?

[patrick-knight]

https://github.com/orgs/community/discussions/118843#discussioncomment-9182731

[figi44]

Stumbled upon push rulesets when looking for a way to prevent changes to workflow definitions across all repos in an org. It also applies to public repos if you configure the rule on org level, but you can easily bypass the rule through a pull request. So pushing a change to a workflow directly to main is denied, but merging a PR with the same change from a fork is allowed, and it doesn't even appear in the Insights logs.

I understand your concerns about open source forks and limiting this feature to push/PR merge for public repos might be a good middle ground. I'm curious about your thoughts on this.

[patrick-knight]

Hey @figi44 thanks for the patience while I replied here.

One thing to clarify is that push rules are not applicable to any public repos. There is some work we need to complete in the beta to ensure that message is consistent in the documentation as well as the UI, as there are a few area where that distinction is not clear.

As far as a future state on full public repos support, I'd be interested to understand your "security" perspective on public repos and how you'd like the interaction with contributors on your public repos? So for any files/path you'd want a push rule for is there an expectation that a contribution for those could come from a fork or are they "sensitive" enough you'd generally never accept that PR

[electrofelix]

Encountered a couple issues when look to add a push ruleset to a repository.

Would like to prevent most contributors from being able to modify certain files except for members of one team and a bot user. I can add both to by-pass the rule however when one of these restricted files has been updated on the main branch, anyone contributing changes via PRs that merges the updated main into their branch it will trigger the rule when they try to push it up.

While we could ask people to rebase, this typically negatively impacts reviewing as once a review as begun rebasing means it's difficult for the review to see just what has changed since their last review.

Additionally the second problem is that when developers hit this rule, there is no option to provide a customised message to help explain how to self resolve.

Would it be possible to ignore commits containing changes to files that should be blocked if the commit is already part of the repository history? e.g.

• default branches
• any protected branch
• target branch

The PR doesn't even show the file as being changed, it's only if you know how to use git diff to compare what was merged into the PR branch from the mainline that it is possible to see what triggered the failure.

Even with such an exclusion, being able to customise the message for failures would be useful. We have a developer portal internally that teams can request changes that would result in generating the files that should not be pushed directly by them. This automatically ensures that the files have correct information that cannot be inferred by pipelines, so while we want developers to be able to trigger the creation of such protected files, we would want to tell them to use the portal instead of them hitting an error and then needing to ask for assistance.

[patrick-knight]

👀 thanks for the reminder to follow up here.

This is a bug we are working to resolve. So when a commit is cherry picked or merged, associated blobs included in these commits are still considered in push rule evaluation, even when unchanged. Which as you point out makes it hard to fully adopt push rules. It's on the short list of things we are resolving before making push rules generally available.

[technologik]

@patrick-knight do we happen to have a date for when GA is expected? I didn't see a date on the Github Public Roadmap after a cursory search. Knowing this would help us with project prioritization in this space. Thanks!

[patrick-knight]

We're targeting August. Don't have an exact date yet, but likely mid-month.

[patrick-knight]

@electrofelix and @technologik I wanted to follow up on this thread as we rolled out some changes recently that should alleviate the concerns. Let me know one way or the other if you are seeing issues.

[electrofelix]

Hi, just wanted to give some feedback that this is working well for us now. Had the ruleset in evaluate for a while, and then active for over a month now and haven't encountered any unexpected issues.

[hauntsaninja]

Restrict file size is awesome and has made life much better, but it would be nice to be able to allowlist specific files

[hauntsaninja]

Would an allowlist glob for large files be a reasonable feature request? @patrick-knight

[patrick-knight]

Its depends what you mean by that? Is there a specific example you are thinking about?

[hauntsaninja]

For instance, in one of our repos, we'd like to have strict file size limits in general, but we have a couple really massive javascript lock files that are somewhat frequently updated. We'd like for the push rule to not reject updates to just those large lock files.

[hauntsaninja]

@patrick-knight what did you think of this feature request?

[patrick-knight]

For that specific case delegated bypass which is a beta for push rules can help there, as it gives a way for a manually gated check.

There's some deeper nuance we need to explore internally about how rules can be bypassed for paths.

[TWiStErRob]

Please link to documentation (like in search) for the syntax allowed in globs:

Context: I want to prevent recreating a deleted folder any of these could be valid, according to all the globbing variants out there:

• my/folder
• my/folder/
• my/folder/*
• my/folder/**
• my/folder/*/**
• my/folder/**/*

but I don't know which of these will be correct.

I found https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#restrict-file-paths, but I needed to search for 3 minutes to find this page, and then I'm redirected to a "technically correct", but not user friendly Ruby documentation.

[TWiStErRob]

As a follow-up on glob docs comment, I tried my/folder/** according to Ruby docs:

Matches directories recursively or files expansively.

and it happily accepted my file:

experiments % git checkout -b my-pr origin/main
experiments % mkdir -p my/folder/foo
experiments % touch my/folder/foo/bar.baz
experiments % git add my/
experiments % git commit -m "Add file"
[my-pr 641d1f3] Add file
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 my/folder/foo/bar.baz
experiments % git push origin HEAD:my-pr -u
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 16 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (6/6), 657 bytes | 657.00 KiB/s, done.
Total 6 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
remote:
remote: Create a pull request for 'my-pr' on GitHub by visiting:
remote:      https://github.com/.../experiments/pull/new/my-pr
remote:
To github.com:.../experiments.git
 * [new branch]      HEAD -> my-pr
branch 'my-pr' set up to track 'origin/my-pr'.

[patrick-knight]

I saw the post from you above this one, but will consolidate my reply here.

Agreed, we need to drive clarity in the UI to make sure you rule works right the first time without manually testing. We're thinking about ways to do that well inclusive of links to docs.

As far as the specifics behavior here. my/folder/** will only block file paths directly underneath it (i.e. my/folder/foo.bar) but not nested files (i.e. my/folder/foo/bar.baz). To accomplish that **/* is the fnmatch pattern needed, so my/folder/**/* will recursively matches all file paths underneath my/folder.

[bo0tzz]

I really like how this feature is shaping up!

We occasionally have the need for an admin to (force-)push to our protected main branch. I was hoping to be able to use the delegated bypass flow to be able to do that with an approval from another admin for some more control, but upon further reading I think it doesn't quite work like that, right?

[patrick-knight]

At the moment it's an experience for push rules and secret scanning bypass, which were good areas for us to start with. Because blocking pushes has a lot of friction and we wanted to be mindful of that. We also know there is a lot of utility beyond those two use cases, but want to learn from those area first and then we'll see where we go next.

[peter-loftus-fbpm]

Is it possible to allow files from the default branch to be compared against so when someone merges it into a feature branch the blocked file extensions can still be merged?

Maybe compare the hash of the files in the default branch and the hash of the file in the commit and if they match allow it.

[patrick-knight]

Push rules are designed to only look for changes to a files based on the git object ID. There are a few scenarios I mentioned here where that is not always true, but we are making head way on resolving that behavior.

[dan-marra-ebrs]

It would be nice if this could be used to restrict merges to the target branches to a list of refs. For instance, if one were trying to enforce gitflow, being able to set something that only allows merges into main from develop. some option like "restrict source branch"

[patrick-knight]

The logic and enforcement for push rules happen before any merge evaluations occur. That said we've gotten feedback on that behavior for branch rules, but have not prioritized any work around that.

[arvid220u]

Would love more flexibility in the rule sets.

What we want: do not allow any updates to folder foo/bar/** on our main branch, unless committed to a PR that's reviewed by an admin.

Currently, it seems like we can either (1) restrict any updates to folder foo/bar/** on all branches (which is not what we want), or (2) restrict any updates to branch main for all folders (which is also not what we want).

Let me know if I'm missing something!

One thing that would be very cool, especially considering that all GitHub users are developers, would be to let users define their protection rules in a simple DSL pushed to .github/rulesets.dsl or something like that. Then, developers could implement a shouldAllow function which takes in all information (branch, committer, files changed, has been reviewed, etc) and just returns a true/false. I think this flexibility would be both easier for you all to implement (since you wouldn't need to do any complex UI work) and also a much better experience for everyone who uses GitHub, since now any rule that anyone could imagine could then be implemented and enforced.

[patrick-knight]

CODEOWNERS with branch&tag rulesets allows you to ensure certain paths have reviewers.

You can achieve a lot of that dsl style customization today with with GitHub actions. By requiring a status check in branch and tag rulesets there is a myriad of decisions you can make and actions workflows are stored as YAML.

especially considering that all GitHub users are developers

Just wanted to mention there are plenty of folks who use GitHub on a daily basis that have range of skills and backgrounds that may not consider themselves developers just yet.

[arvid220u]

Thank you for suggesting the branch&tag ruleset! Unfortunately, it doesn't quite work for us (unless I'm missing something), since the only way that I see how to block CODEOWNERS-files is to require a pull request for all commits, and then turn on "require review from code owners". We do not want to require a pull request for commits touching non-CODEOWNERS files.

For the same reason, I think the GitHub actions approach does not work for us (since we want to allow straight pushes to main for most files).

I get that this might be a niche use case :). Which is why we'd love more flexibility in the rules!