apply_requirements are ignored when policy check is enabled and passes #2182

edbighead · 2022-04-05T13:03:54Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

Atlantis allows apply when mergeable is not satisfied (Github).

Reproduction Steps

Set the following in your repos.yaml config

repos:
  - id: github.com/xxx/yyy
    apply_requirements: ["mergeable"]
    allowed_overrides: [workflow]
    allow_custom_workflows: false
policies:
  owners:
    users:
      - your-user
  policy_sets:
    - name: common-policies
      path: /home/atlantis/policies
      source: local

Add some dummy policies
Add CODEOWNERS file to your github repo and set Require review from Code Owners branch protection
Open a PR and let a non-CODEOWNER approve PR
Comment atlantis apply

Even though mergeable condition is not being satisfied and Github API is returning "mergeable_state": "blocked" from get-a-pull-request, the apply goes through.

Potentially caused by policies_passed in apply_requirements.

Merge check is happening here:

atlantis/server/events/apply_requirement_handler.go

Lines 31 to 34 in 7199ea4

    
           case raw.MergeableApplyRequirement: 
        
           	if !ctx.PullReqStatus.Mergeable { 
        
           		return "Pull request must be mergeable before running apply.", nil 
        
           	}

Mergeability status is fetched from Github API here:

atlantis/server/events/vcs/github_client.go

Lines 299 to 319 in 7199ea4

    
           // PullIsMergeable returns true if the pull request is mergeable. 
        
           func (g *GithubClient) PullIsMergeable(repo models.Repo, pull models.PullRequest) (bool, error) { 
        
           	githubPR, err := g.GetPullRequest(repo, pull.Num) 
        
           	if err != nil { 
        
           		return false, errors.Wrap(err, "getting pull request") 
        
           	} 
        
           	state := githubPR.GetMergeableState() 
        
           	// We map our mergeable check to when the GitHub merge button is clickable. 
        
           	// This corresponds to the following states: 
        
           	// clean: No conflicts, all requirements satisfied. 
        
           	//        Merging is allowed (green box). 
        
           	// unstable: Failing/pending commit status that is not part of the required 
        
           	//           status checks. Merging is allowed (yellow box). 
        
           	// has_hooks: GitHub Enterprise only, if a repo has custom pre-receive 
        
           	//            hooks. Merging is allowed (green box). 
        
           	// See: https://github.com/octokit/octokit.net/issues/1763 
        
           	if state != "clean" && state != "unstable" && state != "has_hooks" { 
        
           		return false, nil 
        
           	} 
        
           	return true, nil 
        
           }

Logs

Obfuscated org and repo names.
Automerge is set to true, apply goes through regardless of mergeable status. Waiting on code owner review from ... log entry tells indeed that this PR is not mergeable.


{"level":"debug","ts":"2022-04-05T14:18:54.388Z","caller":"vcs/github_client.go:277","msg":"GET /repos/xxx/yyy/pulls/4025/reviews","json":{}}
{"level":"debug","ts":"2022-04-05T14:18:55.774Z","caller":"events/project_command_builder.go:547","msg":"Merging config for project at dir: \"terraform/prod/s3/images\" workspace: \"default\"","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:231","msg":"MergeProjectCfg started","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:471","msg":"setting apply_requirements: [mergeable,policies_passed] from repos[2], id: github.com/xxx/yyy","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:471","msg":"setting workflow: \"encrypted\" from repos[2], id: github.com/xxx/yyy","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:471","msg":"setting allowed_overrides: [workflow] from repos[2], id: github.com/xxx/yyy","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:471","msg":"setting allow_custom_workflows: false from repos[2], id: github.com/xxx/yyy","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:471","msg":"setting delete_source_branch_on_merge: false from default server config","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:279","msg":"MergeProjectCfg completed","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"valid/global_cfg.go:282","msg":"final settings: apply_requirements: [mergeable,policies_passed], workflow: encrypted","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"events/project_command_context_builder.go:110","msg":" are enabled","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:55.775Z","caller":"events/project_command_context_builder.go:53","msg":"Building project command context for apply","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"info","ts":"2022-04-05T14:18:56.139Z","caller":"runtime/apply_step_runner.go:38","msg":"starting apply","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:18:56.140Z","caller":"terraform/terraform_client.go:407","msg":"starting \"/usr/local/bin/terraform0.12.31 apply -input=false \\\"/home/atlantis/.atlantis/repos/xxx/yyy/4025/default/terraform/prod/s3/images/tf-prod-s3-images-default.tfplan\\\"\" in \"/home/atlantis/.atlantis/repos/xxx/yyy/4025/default/terraform/prod/s3/images\"","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"info","ts":"2022-04-05T14:18:59.932Z","caller":"terraform/terraform_client.go:467","msg":"successfully ran \"/usr/local/bin/terraform0.12.31 apply -input=false \\\"/home/atlantis/.atlantis/repos/xxx/yyy/4025/default/terraform/prod/s3/images/tf-prod-s3-images-default.tfplan\\\"\" in \"/home/atlantis/.atlantis/repos/xxx/yyy/4025/default/terraform/prod/s3/images\"","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"info","ts":"2022-04-05T14:18:59.932Z","caller":"runtime/apply_step_runner.go:57","msg":"apply successful, deleting planfile","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:19:00.337Z","caller":"vcs/github_client.go:209","msg":"GET /repos/xxx/yyy/issues/4025/comments","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:00.586Z","caller":"vcs/github_client.go:196","msg":"POST /repos/xxx/yyy/issues/4025/comments","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.487Z","caller":"events/db_updater.go:24","msg":"updating DB with pull results","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:19:01.941Z","caller":"vcs/github_client.go:196","msg":"POST /repos/xxx/yyy/issues/4025/comments","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.949Z","caller":"server/middleware.go:44","msg":"POST /events – from 172.1.1.1:21382","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.949Z","caller":"events/events_controller.go:96","msg":"handling GitHub post","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.949Z","caller":"events/events_controller.go:146","msg":"request valid","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.950Z","caller":"events/events_controller.go:152","msg":"handling as comment event","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.950Z","caller":"logging/simple_logger.go:155","msg":"Ignoring non-command comment: \"Ran Apply for project: `tf-prod-s3-image...\"","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:01.950Z","caller":"server/middleware.go:70","msg":"POST /events – respond HTTP 200","json":{}}
{"level":"info","ts":"2022-04-05T14:19:02.895Z","caller":"events/automerger.go:31","msg":"automerging pull request","json":{"repo":"xxx/yyy","pull":"4025"}}
{"level":"debug","ts":"2022-04-05T14:19:02.895Z","caller":"vcs/github_client.go:403","msg":"GET /repos/xxx/yyy","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.206Z","caller":"vcs/github_client.go:426","msg":"PUT /repos/github.User{Login:\"xxx\", ID:6919294, NodeID:\"qqqqq=\", AvatarURL:\"https://avatars.githubusercontent.com/u/xxxxx?v=4\", HTMLURL:\"https://github.com/xxx\", GravatarID:\"\", Type:\"Organization\", SiteAdmin:false, URL:\"https://api.github.com/users/xxx\", EventsURL:\"https://api.github.com/users/xxx/events{/privacy}\", FollowingURL:\"https://api.github.com/users/xxx/following{/other_user}\", FollowersURL:\"https://api.github.com/users/xxx/followers\", GistsURL:\"https://api.github.com/users/xxx/gists{/gist_id}\", OrganizationsURL:\"https://api.github.com/users/xxx/orgs\", ReceivedEventsURL:\"https://api.github.com/users/xxx/received_events\", ReposURL:\"https://api.github.com/users/xxx/repos\", StarredURL:\"https://api.github.com/users/xxx/starred{/owner}{/repo}\", SubscriptionsURL:\"https://api.github.com/users/xxx/subscriptions\"}/0xc00082d380/pulls/4025/merge","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.248Z","caller":"server/middleware.go:44","msg":"POST /events – from 172.1.1.1:54102","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.248Z","caller":"events/events_controller.go:96","msg":"handling GitHub post","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.248Z","caller":"events/events_controller.go:146","msg":"request valid","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.248Z","caller":"events/events_controller.go:152","msg":"handling as comment event","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.248Z","caller":"logging/simple_logger.go:155","msg":"Ignoring non-command comment: \"Automatically merging because all plans ...\"","json":{}}
{"level":"debug","ts":"2022-04-05T14:19:03.248Z","caller":"server/middleware.go:70","msg":"POST /events – respond HTTP 200","json":{}}
{"level":"error","ts":"2022-04-05T14:19:03.904Z","caller":"events/automerger.go:37","msg":"automerging failed: merging pull request: PUT https://api.github.com/repos/xxx/yyy/pulls/4025/merge: 405 Waiting on code owner review from xxx/platform. []","json":{"repo":"xxx/yyy","pull":"4025"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*AutoMerger).automerge\n\tgithub.com/runatlantis/atlantis/server/events/automerger.go:37\ngithub.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/apply_command_runner.go:163\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:259"}
{"level":"debug","ts":"2022-04-05T14:19:03.904Z","caller":"vcs/github_client.go:196","msg":"POST /repos/xxx/yyy/issues/4025/comments","json":{}}

Environment details

Atlantis v0.19.2

Additional Context

I was thoroughly reading the docs to get the answer but no luck. Spotted the error in docs #2170

The text was updated successfully, but these errors were encountered:

edbighead · 2022-04-05T15:34:46Z

when apply_requirements is set to ["approved","mergeable"], the approved check is honored.

grimm26 · 2022-04-05T20:44:57Z

I just noticed this happening and came to check if anyone else was having this issue. Same thing is happening to me, but I am not using policies. I can atlantis apply with no approvals and PR still in not mergable state.

This feels like #1895 all over again

grimm26 · 2022-04-05T20:47:28Z

I wonder if #2053 is causing this? I need to start rolling back versions and seeing when atlantis starts working correctly again.

grimm26 · 2022-04-05T21:17:27Z

I rolled back to 0.19.1 and could still apply a PR with no approvals, not mergable. Thinking that the culprit is #2053, I skipped trying 0.19.0 and rolled back to 0.18.5 and things work again, it doesn't let me apply a non-mergable PR with no approvals.

FYI, I'm using Github Enterprise.

edbighead · 2022-04-06T06:38:22Z

@grimm26 thanks for confirming!
The revert was merged a while ago but not released yet #2173, theoretically the latest dev image should work

edbighead · 2022-04-06T13:12:27Z

tested on latest 7199ea4, works as expected

edbighead added the bug Something isn't working label Apr 5, 2022

edbighead changed the title ~~Allow apply even though not all apply_requirements are being satisfied~~ apply_requirements are ignored when policy check is enabled and passes Apr 5, 2022

edbighead closed this as completed Apr 6, 2022

edbighead mentioned this issue Apr 7, 2022

feat: pending apply status #2188

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apply_requirements are ignored when policy check is enabled and passes #2182

apply_requirements are ignored when policy check is enabled and passes #2182

edbighead commented Apr 5, 2022 •

edited

Loading

edbighead commented Apr 5, 2022

grimm26 commented Apr 5, 2022

grimm26 commented Apr 5, 2022

grimm26 commented Apr 5, 2022

edbighead commented Apr 6, 2022

edbighead commented Apr 6, 2022

apply_requirements are ignored when policy check is enabled and passes #2182

apply_requirements are ignored when policy check is enabled and passes #2182

Comments

edbighead commented Apr 5, 2022 • edited Loading

Community Note

Overview of the Issue

Reproduction Steps

Logs

Environment details

Additional Context

edbighead commented Apr 5, 2022

grimm26 commented Apr 5, 2022

grimm26 commented Apr 5, 2022

grimm26 commented Apr 5, 2022

edbighead commented Apr 6, 2022

edbighead commented Apr 6, 2022

edbighead commented Apr 5, 2022 •

edited

Loading