Skip to content

[v3-0-test] Fix conditionals for test types and make them consistent (#54678) #54691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
potiuk merged 1 commit into from
Aug 20, 2025
Merged

[v3-0-test] Fix conditionals for test types and make them consistent (#54678) #54691

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/basic-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ on: # yamllint disable-line rule:truthy
description: "Whether to run WWW tests (true/false)"
required: true
type: string
needs-api-codegen:
run-api-codegen:
description: "Whether to run API codegen (true/false)"
required: true
type: string
Expand Down
186 changes: 90 additions & 96 deletions .github/workflows/ci-amd.yml
View file
Open in desktop

Large diffs are not rendered by default.

47 changes: 20 additions & 27 deletions .github/workflows/ci-arm.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ jobs:
outputs:
all-python-versions-list-as-string: >-
${{ steps.selective-checks.outputs.all-python-versions-list-as-string }}
amd-runners: ${{ steps.selective-checks.outputs.amd-runners }}
arm-runners: ${{ steps.selective-checks.outputs.arm-runners }}
basic-checks-only: ${{ steps.selective-checks.outputs.basic-checks-only }}
canary-run: ${{ steps.source-run-info.outputs.canary-run }}
ci-image-build: ${{ steps.selective-checks.outputs.ci-image-build }}
Expand Down Expand Up @@ -85,11 +87,6 @@ jobs:
mypy-checks: ${{ steps.selective-checks.outputs.mypy-checks }}
mysql-exclude: ${{ steps.selective-checks.outputs.mysql-exclude }}
mysql-versions: ${{ steps.selective-checks.outputs.mysql-versions }}
needs-api-codegen: ${{ steps.selective-checks.outputs.needs-api-codegen }}
needs-api-tests: ${{ steps.selective-checks.outputs.needs-api-tests }}
needs-helm-tests: ${{ steps.selective-checks.outputs.needs-helm-tests }}
needs-mypy: ${{ steps.selective-checks.outputs.needs-mypy }}
only-new-ui-files: ${{ steps.selective-checks.outputs.only-new-ui-files }}
postgres-exclude: ${{ steps.selective-checks.outputs.postgres-exclude }}
postgres-versions: ${{ steps.selective-checks.outputs.postgres-versions }}
prod-image-build: ${{ steps.selective-checks.outputs.prod-image-build }}
Expand All @@ -101,18 +98,20 @@ jobs:
pull-request-labels: ${{ steps.source-run-info.outputs.pr-labels }}
python-versions-list-as-string: ${{ steps.selective-checks.outputs.python-versions-list-as-string }}
python-versions: ${{ steps.selective-checks.outputs.python-versions }}
run-amazon-tests: ${{ steps.selective-checks.outputs.run-amazon-tests }}
run-airflow-ctl-tests: ${{ steps.selective-checks.outputs.run-airflow-ctl-tests }}
run-amazon-tests: ${{ steps.selective-checks.outputs.run-amazon-tests }}
run-api-codegen: ${{ steps.selective-checks.outputs.run-api-codegen }}
run-api-tests: ${{ steps.selective-checks.outputs.run-api-tests }}
run-coverage: ${{ steps.source-run-info.outputs.run-coverage }}
run-kubernetes-tests: ${{ steps.selective-checks.outputs.run-kubernetes-tests }}
run-task-sdk-tests: ${{ steps.selective-checks.outputs.run-task-sdk-tests }}
run-go-sdk-tests: ${{ steps.selective-checks.outputs.run-go-sdk-tests }}
run-helm-tests: ${{ steps.selective-checks.outputs.run-helm-tests }}
run-kubernetes-tests: ${{ steps.selective-checks.outputs.run-kubernetes-tests }}
run-mypy: ${{ steps.selective-checks.outputs.run-mypy }}
run-system-tests: ${{ steps.selective-checks.outputs.run-system-tests }}
run-tests: ${{ steps.selective-checks.outputs.run-tests }}
run-task-sdk-tests: ${{ steps.selective-checks.outputs.run-task-sdk-tests }}
run-ui-tests: ${{ steps.selective-checks.outputs.run-ui-tests }}
run-unit-tests: ${{ steps.selective-checks.outputs.run-unit-tests }}
run-www-tests: ${{ steps.selective-checks.outputs.run-www-tests }}
amd-runners: ${{ steps.selective-checks.outputs.amd-runners }}
arm-runners: ${{ steps.selective-checks.outputs.arm-runners }}
selected-providers-list-as-string: >-
${{ steps.selective-checks.outputs.selected-providers-list-as-string }}
skip-prek-hooks: ${{ steps.selective-checks.outputs.skip-prek-hooks }}
Expand Down Expand Up @@ -168,7 +167,7 @@ jobs:
runners: ${{ needs.build-info.outputs.arm-runners }}
run-ui-tests: ${{needs.build-info.outputs.run-ui-tests}}
run-www-tests: ${{needs.build-info.outputs.run-www-tests}}
needs-api-codegen: ${{needs.build-info.outputs.needs-api-codegen}}
run-api-codegen: ${{needs.build-info.outputs.run-api-codegen}}
default-python-version: "${{ needs.build-info.outputs.default-python-version }}"
basic-checks-only: ${{needs.build-info.outputs.basic-checks-only}}
skip-prek-hooks: ${{needs.build-info.outputs.skip-prek-hooks}}
Expand Down Expand Up @@ -282,7 +281,7 @@ jobs:
default-python-version: "${{ needs.build-info.outputs.default-python-version }}"
use-uv: ${{ needs.build-info.outputs.use-uv }}
if: >
needs.build-info.outputs.needs-helm-tests == 'true' &&
needs.build-info.outputs.run-helm-tests == 'true' &&
needs.build-info.outputs.default-branch == 'main' &&
needs.build-info.outputs.latest-versions-only != 'true'

Expand Down Expand Up @@ -313,7 +312,7 @@ jobs:
skip-providers-tests: ${{ needs.build-info.outputs.skip-providers-tests }}
use-uv: ${{ needs.build-info.outputs.use-uv }}
default-branch: ${{ needs.build-info.outputs.default-branch }}
if: needs.build-info.outputs.run-tests == 'true'
if: needs.build-info.outputs.run-unit-tests == 'true'

tests-postgres-providers:
name: "Postgres tests: providers"
Expand Down Expand Up @@ -342,7 +341,7 @@ jobs:
skip-providers-tests: ${{ needs.build-info.outputs.skip-providers-tests }}
use-uv: ${{ needs.build-info.outputs.use-uv }}
default-branch: ${{ needs.build-info.outputs.default-branch }}
if: needs.build-info.outputs.run-tests == 'true'
if: needs.build-info.outputs.run-unit-tests == 'true'

tests-sqlite-core:
name: "Sqlite tests: core"
Expand Down Expand Up @@ -373,7 +372,7 @@ jobs:
skip-providers-tests: ${{ needs.build-info.outputs.skip-providers-tests }}
use-uv: ${{ needs.build-info.outputs.use-uv }}
default-branch: ${{ needs.build-info.outputs.default-branch }}
if: needs.build-info.outputs.run-tests == 'true'
if: needs.build-info.outputs.run-unit-tests == 'true'

tests-sqlite-providers:
name: "Sqlite tests: providers"
Expand Down Expand Up @@ -404,7 +403,7 @@ jobs:
skip-providers-tests: ${{ needs.build-info.outputs.skip-providers-tests }}
use-uv: ${{ needs.build-info.outputs.use-uv }}
default-branch: ${{ needs.build-info.outputs.default-branch }}
if: needs.build-info.outputs.run-tests == 'true'
if: needs.build-info.outputs.run-unit-tests == 'true'


tests-non-db-core:
Expand Down Expand Up @@ -435,7 +434,7 @@ jobs:
skip-providers-tests: ${{ needs.build-info.outputs.skip-providers-tests }}
use-uv: ${{ needs.build-info.outputs.use-uv }}
default-branch: ${{ needs.build-info.outputs.default-branch }}
if: needs.build-info.outputs.run-tests == 'true'
if: needs.build-info.outputs.run-unit-tests == 'true'

tests-non-db-providers:
name: "Non-DB tests: providers"
Expand Down Expand Up @@ -465,7 +464,7 @@ jobs:
skip-providers-tests: ${{ needs.build-info.outputs.skip-providers-tests }}
use-uv: ${{ needs.build-info.outputs.use-uv }}
default-branch: ${{ needs.build-info.outputs.default-branch }}
if: needs.build-info.outputs.run-tests == 'true'
if: needs.build-info.outputs.run-unit-tests == 'true'

build-prod-images:
name: Build PROD images
Expand Down Expand Up @@ -510,7 +509,7 @@ jobs:
kubernetes-combos: ${{ needs.build-info.outputs.kubernetes-combos }}
if: >
( needs.build-info.outputs.run-kubernetes-tests == 'true' ||
needs.build-info.outputs.needs-helm-tests == 'true')
needs.build-info.outputs.run-helm-tests == 'true')

# tests-go-sdk:
# name: "Go SDK tests"
Expand All @@ -520,10 +519,7 @@ jobs:
# permissions:
# contents: read
# packages: read
# if: >
# ( needs.build-info.outputs.run-go-sdk-tests == 'true' ||
# needs.build-info.outputs.run-tests == 'true' &&
# needs.build-info.outputs.only-new-ui-files != 'true')
# if: needs.build-info.outputs.run-go-sdk-tests == 'true'
# env:
# GITHUB_REPOSITORY: ${{ github.repository }}
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -534,17 +530,14 @@ jobs:
# uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
# with:
# persist-credentials: false
#
# # keep this in sync with go.mod in go-sdk/
# - name: Setup Go
# uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
# with:
# go-version: 1.24
# cache-dependency-path: go-sdk/go.sum
#
# - name: "Cleanup dist files"
# run: rm -fv ./dist/*
#
# - name: Run Go tests
# working-directory: ./go-sdk
# run: go test -v ./...
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/ci-image-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ on: # yamllint disable-line rule:truthy
description: "Platform for the build - 'linux/amd64' or 'linux/arm64'"
required: true
type: string
needs-mypy:
run-mypy:
description: "Whether to run mypy checks (true/false)"
required: true
type: string
Expand Down Expand Up @@ -88,7 +88,7 @@ on: # yamllint disable-line rule:truthy
description: "Whether to build docs (true/false)"
required: true
type: string
needs-api-codegen:
run-api-codegen:
description: "Whether to run API codegen (true/false)"
required: true
type: string
Expand Down Expand Up @@ -209,7 +209,7 @@ jobs:
name: "MyPy checks"
runs-on: ${{ fromJSON(inputs.runners) }}
needs: install-prek
if: inputs.needs-mypy == 'true'
if: inputs.run-mypy == 'true'
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -404,7 +404,7 @@ jobs:
timeout-minutes: 60
name: "Test Python API client"
runs-on: ${{ fromJSON(inputs.runners) }}
if: inputs.needs-api-codegen == 'true'
if: inputs.run-api-codegen == 'true'
env:
BACKEND: "postgres"
BACKEND_VERSION: "${{ inputs.default-postgres-version }}"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-providers.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ on: # yamllint disable-line rule:truthy
providers-compatibility-tests-matrix:
description: >
JSON-formatted array of providers compatibility tests in the form of array of dicts
(airflow-version, python-versions, remove-providers, run-tests)
(airflow-version, python-versions, remove-providers, run-unit-tests)
required: true
type: string
providers-test-types-list-as-strings-in-json:
Expand Down Expand Up @@ -221,7 +221,7 @@ jobs:
Airflow ${{ matrix.compat.airflow-version }}:Python ${{ matrix.compat.python-version }}
# We do not need to run import check if we run tests, the tests should cover all the import checks
# automatically
if: matrix.compat.run-tests != 'true'
if: matrix.compat.run-unit-tests != 'true'
env:
AIRFLOW_VERSION: "${{ matrix.compat.airflow-version }}"
run: >
Expand All @@ -238,7 +238,7 @@ jobs:
- name: >
Run provider unit tests on Airflow ${{ matrix.compat.airflow-version }}:Python
${{ matrix.compat.python-version }}:${{ matrix.test-types.description }}
if: matrix.compat.run-tests == 'true'
if: matrix.compat.run-unit-tests == 'true'
env:
PROVIDERS_TEST_TYPES: "${{ matrix.test-types.test_types }}"
AIRFLOW_VERSION: "${{ matrix.compat.airflow-version }}"
Expand Down
15 changes: 6 additions & 9 deletions dev/breeze/doc/ci/04_selective_checks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,13 +205,6 @@ GitHub Actions to pass the list of parameters to a command to execute
| mypy-checks | List of folders to be considered for mypy checks | \["airflow_aux", "airflow_core"\] | |
| mysql-exclude | Which versions of MySQL to exclude for tests as JSON array | [] | |
| mysql-versions | Which versions of MySQL to use for tests as JSON array | \['8.0'\] | |
| needs-api-codegen | Whether "api-codegen" are needed to run ("true"/"false") | true | |
| needs-api-tests | Whether "api-tests" are needed to run ("true"/"false") | true | |
| needs-helm-tests | Whether Helm tests are needed to run ("true"/"false") | true | |
| needs-javascript-scans | Whether javascript CodeQL scans should be run ("true"/"false") | true | |
| needs-mypy | Whether mypy check is supposed to run in this build | true | |
| needs-python-scans | Whether Python CodeQL scans should be run ("true"/"false") | true | |
| only-new-ui-files | Whether only new UI files are present in the PR ("true"/"false") | false | |
| postgres-exclude | Which versions of Postgres to exclude for tests as JSON array | [] | |
| postgres-versions | Which versions of Postgres to use for tests as JSON array | \['12'\] | |
| prod-image-build | Whether PROD image build is needed | true | |
Expand All @@ -221,16 +214,20 @@ GitHub Actions to pass the list of parameters to a command to execute
| python-versions | List of python versions to use for that build | \['3.9'\] | |
| python-versions-list-as-string | Which versions of MySQL to use for tests as space-separated string | 3.9 | * |
| run-amazon-tests | Whether Amazon tests should be run ("true"/"false") | true | |
| run-api-codegen | Whether "api-codegen" are needed to run ("true"/"false") | true | |
| run-api-tests | Whether "api-tests" are needed to run ("true"/"false") | true | |
| run-helm-tests | Whether Helm tests are needed to run ("true"/"false") | true | |
| run-kubernetes-tests | Whether Kubernetes tests should be run ("true"/"false") | true | |
| run-mypy | Whether mypy check is supposed to run in this build | true | |
| run-system-tests | Whether system tests should be run ("true"/"false") | true | |
| run-task-sdk-tests | Whether Task SDK tests should be run ("true"/"false") | true | |
| run-tests | Whether unit tests should be run ("true"/"false") | true | |
| run-ui-tests | Whether UI tests should be run ("true"/"false") | true | |
| run-unit-tests | Whether unit tests should be run ("true"/"false") | true | |
| run-www-tests | Whether Legacy WWW tests should be run ("true"/"false") | true | |
| amd-runners | List of labels assigned for runners for that build for public AMD runners | \["ubuntu-22.04"\] | |
| arm-runners | List of labels assigned for runners for that build for public ARM runners | \["ubuntu-22.04-arm"\] | |
| selected-providers-list-as-string | List of providers affected when they are selectively affected. | airbyte http | * |
| skip-prek-hooks | Which prek hooks should be skipped during the static-checks run | flynt,identity | |
| skip-prek-hooks | Which prek hooks should be skipped during the static-checks run | flynt,identity | |
| skip-providers-tests | When provider tests should be skipped (on non-main branch or when no provider changes detected) | true | |
| sqlite-exclude | Which versions of Sqlite to exclude for tests as JSON array | [] | |
| testable-core-integrations | List of core integrations that are testable in the build as JSON array | \['celery', 'kerberos'\] | |
Expand Down
6 changes: 3 additions & 3 deletions dev/breeze/src/airflow_breeze/global_constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -750,19 +750,19 @@ def generate_provider_dependencies_if_needed():
"python-version": "3.9",
"airflow-version": "2.10.5",
"remove-providers": "cloudant common.messaging fab git keycloak",
"run-tests": "true",
"run-unit-tests": "true",
},
{
"python-version": "3.9",
"airflow-version": "2.11.0",
"remove-providers": "cloudant common.messaging fab git keycloak",
"run-tests": "true",
"run-unit-tests": "true",
},
{
"python-version": "3.9",
"airflow-version": "3.0.2",
"remove-providers": "cloudant",
"run-tests": "true",
"run-unit-tests": "true",
},
]

Expand Down
Loading
Loading