Atlantis able to apply when apply requirements are set to mergable, and the required checks status are in progress

[Yasmine92]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Atlantis does not behave as expected when the required checks are still in progress, it considers the apply requirement [mergeable] met and proceeds to apply.
with the option auto-merge enabled, it fails after applying to merge because obviously the requirements for mergeability are still not met

Reproduction Steps

Re-run the required checks on your Pull request, and comment atlantis apply while the checks are still in progress.

Logs

Logs

{"level":"debug","ts":"2023-10-04T10:12:30.117Z","caller":"vcs/github_client.go:330","msg":"GET /repos/Yasmine/cloud-services-sandbox/pulls/46/reviews","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:30.870Z","caller":"vcs/github_client.go:504","msg":"AllowMergeableBypassApply feature flag is enabled - attempting to bypass apply from mergeable requirements","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:32.759Z","caller":"events/project_command_builder.go:747","msg":"Merging config for project at dir: \"terraform\" workspace: \"dev\"","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.759Z","caller":"valid/global_cfg.go:301","msg":"MergeProjectCfg started","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting apply_requirements: [mergeable,policies_passed] from repos[1], id: /.*/","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting delete_source_branch_on_merge: false from default server config","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting repo_locking: false from repos[1], id: /.*/","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting policy_check: true from default server config","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting plan_requirements: [policies_passed] from repos[1], id: /.*/","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting import_requirements: [policies_passed] from repos[1], id: /.*/","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting workflow: \"default\" from default server config","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting allowed_overrides: [workflow] from repos[1], id: /.*/","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:591","msg":"setting allow_custom_workflows: true from repos[1], id: /.*/","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:342","msg":"overriding server-defined workflow with repo-specified workflow: \"dev\"","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:369","msg":"MergeProjectCfg completed","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"valid/global_cfg.go:372","msg":"final settings: plan_requirements: [policies_passed], apply_requirements: [mergeable,policies_passed], import_requirements: [policies_passed], workflow: dev","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.761Z","caller":"events/project_command_context_builder.go:170","msg":"PolicyChecks are enabled","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"info","ts":"2023-10-04T10:12:32.763Z","caller":"terraform/terraform_client.go:317","msg":"Cannot determine which version to use from terraform configuration, detected 2 possibilities.","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:32.763Z","caller":"events/project_command_context_builder.go:98","msg":"Building project command context for apply","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"info","ts":"2023-10-04T10:12:32.770Z","caller":"terraform/terraform_client.go:317","msg":"Cannot determine which version to use from terraform configuration, detected 2 possibilities.","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"info","ts":"2023-10-04T10:12:33.137Z","caller":"runtime/apply_step_runner.go:39","msg":"starting apply","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:33.137Z","caller":"models/shell_command_runner.go:95","msg":"starting \"/usr/local/bin/terraform apply -input=false \\\"/atlantis-data/repos/Yasmine/cloud-services-sandbox/46/dev/terraform/dev-dev.tfplan\\\"\" in \"/atlantis-data/repos/Yasmine/cloud-services-sandbox/46/dev/terraform\"","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"info","ts":"2023-10-04T10:12:35.994Z","caller":"models/shell_command_runner.go:161","msg":"successfully ran \"/usr/local/bin/terraform apply -input=false \\\"/atlantis-data/repos/Yasmine/cloud-services-sandbox/46/dev/terraform/dev-dev.tfplan\\\"\" in \"/atlantis-data/repos/Yasmine/cloud-services-sandbox/46/dev/terraform\"","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46","duration":2.857285489}}
{"level":"info","ts":"2023-10-04T10:12:35.994Z","caller":"runtime/apply_step_runner.go:58","msg":"apply successful, deleting planfile","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"info","ts":"2023-10-04T10:12:36.743Z","caller":"events/instrumented_project_command_runner.go:88","msg":"apply success. output available at: https://github.com/Yasmine/cloud-services-sandbox/pull/46","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:36.744Z","caller":"vcs/github_client.go:212","msg":"GET /repos/Yasmine/cloud-services-sandbox/issues/46/comments","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:37.101Z","caller":"vcs/github_client.go:212","msg":"GET /repos/Yasmine/cloud-services-sandbox/issues/46/comments","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:41.535Z","caller":"server/middleware.go:45","msg":"GET /healthz – from 35.191.9.160:46240","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:41.535Z","caller":"server/middleware.go:72","msg":"GET /healthz – respond HTTP 200","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:41.800Z","caller":"server/middleware.go:45","msg":"GET /healthz – from 10.16.0.7:52652","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:41.802Z","caller":"server/middleware.go:72","msg":"GET /healthz – respond HTTP 200","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:42.311Z","caller":"vcs/github_client.go:192","msg":"POST /repos/Yasmine/cloud-services-sandbox/issues/46/comments","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:43.302Z","caller":"events/db_updater.go:25","msg":"updating DB with pull results","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:43.701Z","caller":"server/middleware.go:45","msg":"POST /events – from 10.16.0.2:51282","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:43.701Z","caller":"events/events_controller.go:103","msg":"handling GitHub post","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:43.716Z","caller":"events/events_controller.go:169","msg":"request valid","json":{"gh-request-id":"X-Github-Delivery=8e6a4cb0-629e-11ee-8549-27d0124367c1"}}
{"level":"debug","ts":"2023-10-04T10:12:43.716Z","caller":"server/middleware.go:72","msg":"POST /events – respond HTTP 200","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:43.720Z","caller":"vcs/github_client.go:192","msg":"POST /repos/Yasmine/cloud-services-sandbox/issues/46/comments","json":{}}
{"level":"info","ts":"2023-10-04T10:12:44.661Z","caller":"events/automerger.go:32","msg":"automerging pull request","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}
{"level":"debug","ts":"2023-10-04T10:12:44.661Z","caller":"vcs/github_client.go:584","msg":"GET /repos/Yasmine/cloud-services-sandbox","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:44.909Z","caller":"vcs/github_client.go:607","msg":"PUT /repos/github.User{Login:\"Yasmine\", ID:XXXXXXXXXX, NodeID:\"XXXXXXXXXXXXXXXXXXXXXXXX\", AvatarURL:\"https://avatars.githubusercontent.com/u/19650282?v=4\", HTMLURL:\"https://github.com/Yasmine\", GravatarID:\"\", Type:\"Organization\", SiteAdmin:false, URL:\"https://api.github.com/users/Yasmine\", EventsURL:\"https://api.github.com/users/Yasmine/events{/privacy}\", FollowingURL:\"https://api.github.com/users/Yasmine/following{/other_user}\", FollowersURL:\"https://api.github.com/users/Yasmine/followers\", GistsURL:\"https://api.github.com/users/Yasmine/gists{/gist_id}\", OrganizationsURL:\"https://api.github.com/users/Yasmine/orgs\", ReceivedEventsURL:\"https://api.github.com/users/Yasmine/received_events\", ReposURL:\"https://api.github.com/users/Yasmine/repos\", StarredURL:\"https://api.github.com/users/Yasmine/starred{/owner}{/repo}\", SubscriptionsURL:\"https://api.github.com/users/Yasmine/subscriptions\"}/0xc000ce7020/pulls/46/merge","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:45.090Z","caller":"server/middleware.go:45","msg":"POST /events – from 35.191.13.215:62925","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:45.090Z","caller":"events/events_controller.go:103","msg":"handling GitHub post","json":{}}
{"level":"debug","ts":"2023-10-04T10:12:45.092Z","caller":"events/events_controller.go:169","msg":"request valid","json":{"gh-request-id":"X-Github-Delivery=8f39aaa0-629e-11ee-8088-b6da67cc182b"}}
{"level":"debug","ts":"2023-10-04T10:12:45.093Z","caller":"server/middleware.go:72","msg":"POST /events – respond HTTP 200","json":{}}
{"level":"error","ts":"2023-10-04T10:12:45.562Z","caller":"vcs/instrumented_client.go:252","msg":"Unable to merge pull, error: merging pull request: PUT https://api.github.com/repos/Yasmine/cloud-services-sandbox/pulls/46/merge: 405 Required status check \"pre-commit\" is in progress. []","json":{"pull-num":46},"stacktrace":"github.com/runatlantis/atlantis/server/events/vcs.(*InstrumentedClient).MergePull\n\tgithub.com/runatlantis/atlantis/server/events/vcs/instrumented_client.go:252\ngithub.com/runatlantis/atlantis/server/events/vcs.(*ClientProxy).MergePull\n\tgithub.com/runatlantis/atlantis/server/events/vcs/proxy.go:88\ngithub.com/runatlantis/atlantis/server/events.(*AutoMerger).automerge\n\tgithub.com/runatlantis/atlantis/server/events/automerger.go:35\ngithub.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/apply_command_runner.go:183\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:301"}
{"level":"debug","ts":"2023-10-04T10:12:45.562Z","caller":"events/post_workflow_hooks_command_runner.go:60","msg":"post-hooks configured, running...","json":{"repo":"Yasmine/cloud-services-sandbox","pull":"46"}}

Environment details

Atlantis server-side config file:

---
repos:
- id: /.*/
  allowed_overrides: [workflow]
  allow_custom_workflows: true
  apply_requirements: [mergeable]
  pre_workflow_hooks:
    - run: python /home/atlantis/github_auto --hook=pre
  post_workflow_hooks:
    - run: python /home/atlantis/github_auto --hook=post
    - run: rm -rf /tmp/$BASE_REPO_OWNER-$BASE_REPO_NAME-$PULL_NUM
  repo_locking: false
############################################################################
# Custom workflows: workflows that can be directly called from atlantis.yaml
############################################################################
workflows:
  # only terraform, for repositories using default workspace only.
  terraform-default-workspace:
    plan:
      steps:
      - init
      - plan
      - show
      - run: python /home/atlantis/github_auto
    import:
      steps:
        - import:
            extra_args: ["-var-file", "environments/$WORKSPACE.tfvars"]
metrics:
  prometheus:
    endpoint: /metrics

Repo atlantis.yaml file:

---
# atlantis.yaml
version: 3
parallel_plan: true
projects:
- name: dev
  dir: terraform
  autoplan:
    when_modified: ["*.tf*", ".terraform.lock.hcl"]
  workflow: dev
  workspace: dev

workflows:
  dev:
    plan:
      steps:
      - init:
      - run: terraform providers lock -platform=linux_amd64
      - plan:
          extra_args: ["-var-file", "environments/dev.tfvars"]

• Atlantis version: atlantis v0.25.0
• Deployment method: helm chart
• If not running the latest Atlantis version have you tried to reproduce this issue on the latest version:
• Atlantis flags: atlantis server --config=/etc/atlantis/atlantis.yaml --enable-diff-markdown-format --automerge --checkout-strategy=merge --hide-prev-plan-comments --enable-policy-checks --gh-allow-mergeable-bypass-apply --quiet-policy-checks --allow-commands=version,plan,apply,unlock,approve_policies,import --silence-no-projects --emoji-reaction=ok_hand

Additional Context

[Yasmine92]

currently working on a PR with a potential fix