Change capture warnings output format to the JSON #38905

Taragolis · 2024-04-10T18:34:23Z

This PR is a result of complete rework on warning capture system:

Move capture warnings system into the separate module, and will be conditionally enable as deferrable plugin
Works with xdists
Output format is JSON rather than pylint (we do not use it anyway).
Pass parameters from the pytest configurations with possibility to disable it. If it disabled than plugin not activated and hooks pytest hooks injected.
Keep information about original warning and in which test suite it originally detected
Output path now relative to the tests directory if non -absolute path provided, but default stored into the tests/warnings.txt regardless of the current working directory.
Trim warning message filename paths to the project root or sites (system and users). Also add groups in which type of files warning raises. This one depends on stacklevel if it set incorrectly than it might come to the non-expected category
- providers - starts with airflow/providers/
- airflow - starts with airflow/
- tests - starts with tests/
- other - raised by third parties or in stdlib, or maybe
Avoid to use internal pytests classes which not a part of public interface
New warning message summariser, and also store merged file and per group into the Github artefacts

The only things the same with pytest-capture-warnings is an idea, so I remove license headers, if some one have any objections feel free to ping me to return it.

Output before

tests/core/test_configuration.py:1411: [W0513(warning), ] The sql_alchemy_conn option in [core] has been moved to the sql_alchemy_conn option in [database] - the old setting has been used, but please update your config.
tests/core/test_core.py:107: [W0513(warning), ] SELECT statement has a cartesian product between FROM element(s) "task_fail" and FROM element "dag_run".  Apply join condition(s) between each element to resolve.
tests/core/test_logging_config.py:267: [W0513(warning), ] The `airflow.utils.log.wasb_task_handler.WasbTaskHandler` class is deprecated. Please use `'airflow.providers.microsoft.azure.log.wasb_task_handler.WasbTaskHandler'`.

Output after

{"category": "DeprecationWarning", "message": "The sql_alchemy_conn option in [core] has been moved to the sql_alchemy_conn option in [database] - the old setting has been used, but please update your config.", "node_id": "tests/core/test_configuration.py::TestDeprecatedConf::test_conf_as_dict_when_deprecated_value_in_secrets", "filename": "tests/core/test_configuration.py", "lineno": 1411, "count": 1}
{"category": "sqlalchemy.exc.SAWarning", "message": "SELECT statement has a cartesian product between FROM element(s) \"task_fail\" and FROM element \"dag_run\".  Apply join condition(s) between each element to resolve.", "node_id": "tests/core/test_core.py::TestCore::test_task_fail_duration", "filename": "tests/core/test_core.py", "lineno": 107, "count": 1}
{"category": "DeprecationWarning", "message": "The `airflow.utils.log.wasb_task_handler.WasbTaskHandler` class is deprecated. Please use `'airflow.providers.microsoft.azure.log.wasb_task_handler.WasbTaskHandler'`.", "node_id": "tests/core/test_logging_config.py::TestLoggingSettings::test_loading_remote_logging_with_wasb_handler", "filename": "tests/core/test_logging_config.py", "lineno": 267, "count": 1}

Sample output of Finalize tests / Summarize warnings

Current Working Directory: /home/runner/work/airflow/airflow
 Process directory /home/runner/work/airflow/airflow/artifacts with pattern '**/warnings-*.txt' 
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Postgres-postgres-12-3.8/warnings-other-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-branchexternalpython-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-branchpythonvenv-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-cli-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-api-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-operators-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-other-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-providers_amazon-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-providers_-amazon_google-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-plainasserts-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-serialization-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-www-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-core-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-pythonvenv-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-providers_google-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MySQL-mysql-8.0-3.8/warnings-always-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-Non-DB--sqlite--3.8/warnings-all-none.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-integration-postgres-12/warnings-integration-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-Removed-Postgres-postgres-12-3.8/warnings-other-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-providers_-amazon_google-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-api-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-plainasserts-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-operators-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-core-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-www-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-other-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-branchpythonvenv-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-pythonvenv-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-cli-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-providers_amazon-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-branchexternalpython-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-always-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-serialization-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-Sqlite-sqlite--3.8/warnings-providers_google-sqlite.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-DB-MinSQLAlchemy-Postgres-postgres-12-3.8/warnings-other-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-LatestBoto-Postgres-postgres-12-3.8/warnings-other-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pendulum2-Postgres-postgres-12-3.8/warnings-other-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-Pydantic-V1-Postgres-postgres-12-3.8/warnings-other-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-integration-mysql-8.0/warnings-integration-mysql.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-pythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-operators-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-api-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-plainasserts-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-branchpythonvenv-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-providers_amazon-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-providers_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-providers_-amazon_google-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-serialization-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-branchexternalpython-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-cli-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-core-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-always-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-www-postgres.txt
Process file: /home/runner/work/airflow/airflow/artifacts/test-warnings-All-InProgressDisabled-Postgres-postgres-12-3.8/warnings-other-postgres.txt
----------------- Total processed lines 71,400 in 138 file(s) ------------------
Good Records: 71,400. Saved into file /home/runner/work/airflow/airflow/files/warn-summary-all.txt
=============================== Group 'airflow' ================================
Saved into file: /home/runner/work/airflow/airflow/files/warn-summary-group-airflow.txt
Unique warnings within the test cases: 544
Top 10 Tests Cases:
  airflow.utils.context.AirflowContextDeprecationWarning tests/cli/commands/test_task_command.py::TestCliTasks::test_test - 12 (!)
  airflow.utils.context.AirflowContextDeprecationWarning tests/cli/commands/test_task_command.py::TestCliTasks::test_test_no_execution_date - 12 (!)
  airflow.utils.context.AirflowContextDeprecationWarning tests/cli/commands/test_task_command.py::TestCliTasks::test_test_with_existing_dag_run - 12 (!)
  airflow.utils.context.AirflowContextDeprecationWarning tests/models/test_mappedoperator.py::test_expand_mapped_task_instance_with_named_index - 12 (!)
  airflow.utils.context.AirflowContextDeprecationWarning tests/operators/test_python.py::TestPythonVirtualenvOperator::test_airflow_context - 12 (!)
  airflow.utils.context.AirflowContextDeprecationWarning tests/operators/test_python.py::TestPythonVirtualenvOperator::test_pendulum_context - 12 (!)
  airflow.utils.context.AirflowContextDeprecationWarning tests/operators/test_python.py::TestPythonVirtualenvOperator::test_base_context - 8 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/cli/commands/test_connection_command.py::TestCliImportConnections::test_cli_connections_import_should_load_connections - 6 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/cli/commands/test_connection_command.py::TestCliImportConnections::test_cli_connections_import_should_not_overwrite_existing_connections - 6 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/cli/commands/test_connection_command.py::TestCliImportConnections::test_cli_connections_import_should_overwrite_existing_connections - 6 (!)
Unique warnings: 94
Warnings grouped by category:
  airflow.exceptions.RemovedInAirflow3Warning - 44 (!)
  airflow.utils.context.AirflowContextDeprecationWarning - 24 (!)
  DeprecationWarning - 16
  airflow.exceptions.AirflowProviderDeprecationWarning - 5 (!)
  UserWarning - 2
  sqlalchemy.exc.SAWarning - 2 (!!)
  FutureWarning - 1
Top 10 Warnings:
  airflow/utils/context.py:240:airflow.utils.context.AirflowContextDeprecationWarning - 12 (!)
  airflow/utils/context.py:348:airflow.utils.context.AirflowContextDeprecationWarning - 12 (!)
  airflow/models/connection.py:375:airflow.exceptions.RemovedInAirflow3Warning - 10 (!)
  airflow/models/connection.py:177:airflow.exceptions.RemovedInAirflow3Warning - 9 (!)
  airflow/models/connection.py:369:airflow.exceptions.RemovedInAirflow3Warning - 9 (!)
  airflow/www/views.py:1743:airflow.exceptions.RemovedInAirflow3Warning - 2 (!)
  airflow/utils/db.py:1069:sqlalchemy.exc.SAWarning - 2 (!!)
  airflow/api_connexion/endpoints/forward_to_fab_endpoint.py:48:DeprecationWarning - 1
  airflow/api_internal/internal_api_call.py:127:DeprecationWarning - 1
  airflow/cli/commands/connection_command.py:165:DeprecationWarning - 1
-------------------------- Always reported warnings 2---------------------------
airflow/utils/db.py:1069
  sqlalchemy.exc.SAWarning - Skipped unsupported reflection of expression-based index idx_ab_user_username
airflow/utils/db.py:1069
  sqlalchemy.exc.SAWarning - Skipped unsupported reflection of expression-based index idx_ab_register_user_username
================================ Group 'other' =================================
Saved into file: /home/runner/work/airflow/airflow/files/warn-summary-group-other.txt
Unique warnings within the test cases: 6,815
Top 10 Tests Cases:
  DeprecationWarning tests/www/views/test_views_connection.py::test_create_connection - 615
  DeprecationWarning tests/cli/commands/test_connection_command.py::TestCliAddConnections::test_cli_connection_add - 614
  DeprecationWarning tests/always/test_example_dags.py::test_should_be_importable - 611
  DeprecationWarning tests/serialization/test_dag_serialization.py::TestStringifiedDAGs::test_serialization - 611
  DeprecationWarning tests/jobs/test_scheduler_job.py::TestSchedulerJob::test_extra_operator_links_not_loaded_in_scheduler_loop - 602
  DeprecationWarning tests/providers/amazon/aws/links/test_athena.py::TestAthenaQueryResultsLink::test_link_serialize - 602
  DeprecationWarning tests/providers/google/cloud/operators/test_bigquery.py::TestBigQueryOperator::test_bigquery_operator_extra_serialized_field_when_single_query - 602
  DeprecationWarning tests/sensors/test_external_task_sensor.py::TestExternalTaskMarker::test_serialized_external_task_marker - 602
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_dagbag.py::TestDagBag::test_load_subdags - 8 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_dagbag.py::TestDagBag::test_skip_cycle_dags - 6 (!)
Unique warnings: 690
Warnings grouped by category:
  DeprecationWarning - 632
  airflow.exceptions.RemovedInAirflow3Warning - 36 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning - 8 (!)
  UserWarning - 6
  RuntimeWarning - 3
  PendingDeprecationWarning - 1
  celery.exceptions.CPendingDeprecationWarning - 1 (!!)
  pydantic.warnings.PydanticDeprecatedSince20 - 1 (!!)
  requests_kerberos.kerberos_.NoCertificateRetrievedWarning - 1
  urllib3.exceptions.InsecureRequestWarning - 1
Top 10 Warnings:
  _pytest/python.py:194:airflow.exceptions.RemovedInAirflow3Warning - 8 (!)
  eventlet/green/OpenSSL/crypto.py:1:DeprecationWarning - 4
  alembic/ddl/impl.py:775:UserWarning - 4
  /usr/local/lib/python3.8/copyreg.py:91:airflow.exceptions.AirflowProviderDeprecationWarning - 4 (!)
  connexion/apis/flask_api.py:236:DeprecationWarning - 2
  impala/interface.py:273:DeprecationWarning - 2
  /usr/local/lib/python3.8/unittest/mock.py:1325:airflow.exceptions.RemovedInAirflow3Warning - 2 (!)
  /usr/local/lib/python3.8/unittest/mock.py:1756:airflow.exceptions.RemovedInAirflow3Warning - 2 (!)
  <string>:1:airflow.exceptions.RemovedInAirflow3Warning - 2 (!)
  sqlalchemy/orm/attributes.py:606:airflow.exceptions.RemovedInAirflow3Warning - 2 (!)
============================== Group 'providers' ===============================
Saved into file: /home/runner/work/airflow/airflow/files/warn-summary-group-providers.txt
Unique warnings within the test cases: 177
Top 10 Tests Cases:
  UserWarning tests/providers/microsoft/azure/hooks/test_adx.py::TestAzureDataExplorerHook::test_backcompat_prefix_both_causes_warning - 5
  UserWarning tests/providers/snowflake/hooks/test_snowflake.py::TestPytestSnowflakeHook::test_extra_short_beats_long - 5
  DeprecationWarning tests/providers/cncf/kubernetes/executors/test_kubernetes_executor.py::TestKubernetesExecutor::test_kube_config_get_namespace_list - 4
  airflow.exceptions.AirflowProviderDeprecationWarning tests/providers/google/cloud/operators/test_vertex_ai.py::TestVertexAIDeleteCustomTrainingJobOperator::test_execute - 4 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning tests/always/test_example_dags.py::test_should_be_importable - 3 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning tests/always/test_example_dags.py::test_should_not_do_database_queries - 3 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning tests/serialization/test_dag_serialization.py::TestStringifiedDAGs::test_serialization - 3 (!)
  DeprecationWarning tests/providers/cncf/kubernetes/executors/test_kubernetes_executor.py::TestKubernetesExecutor::test_get_task_log - 2
  DeprecationWarning tests/providers/google/cloud/operators/test_bigquery.py::TestBigQueryInsertJobOperator::test_execute_openlineage_events - 2
  RuntimeWarning tests/providers/databricks/hooks/test_databricks.py::TestDatabricksHookAsyncAadToken::test_get_run_state - 2
Unique warnings: 74
Warnings grouped by category:
  airflow.exceptions.AirflowProviderDeprecationWarning - 42 (!)
  UserWarning - 14
  DeprecationWarning - 11
  RuntimeWarning - 6
  FutureWarning - 1
Top 10 Warnings:
  airflow/providers/microsoft/azure/hooks/adx.py:163:UserWarning - 5
  airflow/providers/google/common/hooks/base_google.py:486:airflow.exceptions.AirflowProviderDeprecationWarning - 5 (!)
  airflow/providers/jdbc/hooks/jdbc.py:39:UserWarning - 3
  airflow/providers/amazon/aws/hooks/eks.py:79:DeprecationWarning - 1
  airflow/providers/apache/cassandra/hooks/cassandra.py:121:DeprecationWarning - 1
  airflow/providers/cncf/kubernetes/kube_client.py:116:DeprecationWarning - 1
  airflow/providers/cncf/kubernetes/kube_config.py:37:DeprecationWarning - 1
  airflow/providers/cncf/kubernetes/kube_config.py:64:DeprecationWarning - 1
  airflow/providers/cncf/kubernetes/kube_config.py:65:DeprecationWarning - 1
  airflow/providers/cncf/kubernetes/kube_config.py:68:DeprecationWarning - 1
================================ Group 'tests' =================================
Saved into file: /home/runner/work/airflow/airflow/files/warn-summary-group-tests.txt
Unique warnings within the test cases: 759
Top 10 Tests Cases:
  airflow.exceptions.AirflowProviderDeprecationWarning tests/always/test_example_dags.py::test_should_be_importable - 57 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning tests/always/test_example_dags.py::test_should_not_do_database_queries - 57 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning tests/serialization/test_dag_serialization.py::TestStringifiedDAGs::test_serialization - 57 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_dagbag.py::TestDagBag::test_load_subdags - 8 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_dagbag.py::TestDagBag::test_skip_cycle_dags - 6 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_skipmixin.py::TestSkipMixin::test_mapped_tasks_skip_all_except - 6 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/utils/test_dates.py::TestDates::test_days_ago - 6 (!)
  airflow.exceptions.AirflowProviderDeprecationWarning tests/providers/google/cloud/operators/test_mlengine.py::TestMLEngineStartBatchPredictionJobOperator::test_invalid_model_origin - 4 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_dag.py::TestDag::test_clear_set_dagrun_state_for_parent_dag - 4 (!)
  airflow.exceptions.RemovedInAirflow3Warning tests/models/test_dag.py::TestDag::test_clear_set_dagrun_state_for_subdag - 4 (!)
Unique warnings: 609
Warnings grouped by category:
  airflow.exceptions.AirflowProviderDeprecationWarning - 405 (!)
  airflow.exceptions.RemovedInAirflow3Warning - 155 (!)
  DeprecationWarning - 28
  UserWarning - 9
  FutureWarning - 6
  sqlalchemy.exc.SAWarning - 5 (!!)
  pytest.PytestWarning - 1 (!!)
Top 10 Warnings:
  tests/conftest.py:1097:airflow.exceptions.AirflowProviderDeprecationWarning - 17 (!)
  tests/test_utils/config.py:36:FutureWarning - 2
  tests/providers/slack/utils/test_utils.py:60:UserWarning - 2
  tests/providers/amazon/aws/operators/test_eks.py:731:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
  tests/providers/amazon/aws/operators/test_emr_notebook_execution.py:62:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
  tests/providers/amazon/aws/operators/test_emr_notebook_execution.py:94:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
  tests/providers/amazon/aws/operators/test_emr_notebook_execution.py:122:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
  tests/providers/amazon/aws/operators/test_emr_notebook_execution.py:152:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
  tests/providers/amazon/aws/operators/test_emr_notebook_execution.py:187:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
  tests/providers/amazon/aws/operators/test_emr_notebook_execution.py:217:airflow.exceptions.AirflowProviderDeprecationWarning - 2 (!)
-------------------------- Always reported warnings 6---------------------------
tests/cli/commands/test_task_command.py:389
  pytest.PytestWarning - The test <Function test_cli_test_with_deferrable_operator> is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'.
tests/core/test_core.py:107
  sqlalchemy.exc.SAWarning - SELECT statement has a cartesian product between FROM element(s) "task_fail" and FROM element "dag_run".  Apply join condition(s) between each element to resolve.
tests/utils/test_db.py:73
  sqlalchemy.exc.SAWarning - Skipped unsupported reflection of expression-based index idx_ab_user_username
tests/utils/test_db.py:73
  sqlalchemy.exc.SAWarning - Skipped unsupported reflection of expression-based index idx_ab_register_user_username
tests/utils/test_db_cleanup.py:216
  sqlalchemy.exc.SAWarning - Class CreateTableAs will not make use of SQL compilation caching as it does not set the 'inherit_cache' attribute to ``True``.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this object can make use of the cache key generated by the superclass.  Alternatively, this attribute may be set to False which will disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
tests/core/test_core.py:107
  sqlalchemy.exc.SAWarning - SELECT statement has a cartesian product between FROM element(s) "dag_run" and FROM element "task_fail".  Apply join condition(s) between each element to resolve.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

potiuk

LGTM

Taragolis · 2024-04-10T19:56:41Z

Test tests/ti_deps/deps/test_mapped_task_upstream_dep.py::test_step_by_step is so annoying, it failed into the CI on the regular basis in the different PR and I can't still reproduce it locally

Taragolis · 2024-04-10T21:20:55Z

Also add about configuration into the documentation, so worthwhile to check spelling and grammar

Taragolis · 2024-04-11T15:43:03Z

Covert to draft for avoid accidentally merge

I have a plan to change a bit what we store into output file
Pytest output format, e.g. where warning raised airflow, providers, test or other
Add warning summariser in the CI

Warnings output from this PR enough for play with merge into resulting dump, split into the categories and found some unnecessary or important warnings

Many changes since original review

Make it also work in xdist worker and exchange data with controller Provide simple script for summarize warnings with simple report

contributing-docs/testing/unit_tests.rst

Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>

Taragolis · 2024-04-15T17:16:07Z

Feel difference

Python 3.12 (non-db)

================== Warning summary. Total: 4,670, Unique: 329 ==================
airflow: total 133, unique 16
other: total 3,737, unique 24
providers: total 222, unique 44
tests: total 578, unique 245
Warnings saved into /files/warnings-all-none.txt file.

Python 3.11 (non-db)

================== Warning summary. Total: 1,136, Unique: 320 ==================
airflow: total 132, unique 15
other: total 204, unique 17
providers: total 224, unique 45
tests: total 576, unique 243
Warnings saved into /files/warnings-all-none.txt file.

Python 3.8 (non-db)

================== Warning summary. Total: 1,134, Unique: 320 ==================
airflow: total 132, unique 15
other: total 203, unique 18
providers: total 223, unique 44
tests: total 576, unique 243
Warnings saved into /files/warnings-all-none.txt file.

The differences mostly because some upstream dependency still use datetime.datetime.utcnow() (e.g. botocore), some deprecated constant from ast module (werkzeug, should be resolved as soon as we will able to update to newer version)

And also fork from multithreading, which are "deprecated" in Python 3.12: https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555

{"category": "DeprecationWarning", "message": "This process (pid=99) is multi-threaded, use of fork() may lead to deadlocks in the child.", "node_id": "tests/secrets/test_cache.py::TestSecretCache::test_cache_accessible_from_other_process", "filename": "/usr/local/lib/python3.12/multiprocessing/popen_fork.py", "lineno": 66}
{"category": "DeprecationWarning", "message": "This process (pid=99) is multi-threaded, use of fork() may lead to deadlocks in the child.", "node_id": "tests/providers/celery/executors/test_celery_kubernetes_executor.py::TestCeleryKubernetesExecutor::test_queue_command", "filename": "/usr/local/lib/python3.12/multiprocessing/popen_fork.py", "lineno": 66}
{"category": "DeprecationWarning", "message": "This process (pid=99) is multi-threaded, use of fork() may lead to deadlocks in the child.", "node_id": "tests/providers/celery/executors/test_celery_kubernetes_executor.py::TestCeleryKubernetesExecutor::test_job_id_setter", "filename": "/usr/local/lib/python3.12/multiprocessing/popen_fork.py", "lineno": 66}
{"category": "DeprecationWarning", "message": "This process (pid=99) is multi-threaded, use of fork() may lead to deadlocks in the child.", "node_id": "tests/providers/celery/executors/test_celery_kubernetes_executor.py::TestCeleryKubernetesExecutor::test_send_callback", "filename": "/usr/local/lib/python3.12/multiprocessing/popen_fork.py", "lineno": 66}

Taragolis added the canary When set on PR running from apache repo - behave as canary run label Apr 10, 2024

Taragolis requested review from potiuk, uranusjr, ephraimbuddy, hussein-awala, eladkal and vincbeck April 10, 2024 18:34

potiuk previously approved these changes Apr 10, 2024

View reviewed changes

vincbeck previously approved these changes Apr 10, 2024

View reviewed changes

Taragolis closed this Apr 10, 2024

Taragolis reopened this Apr 10, 2024

Taragolis added full tests needed We need to run full set of tests for this PR to merge and removed canary When set on PR running from apache repo - behave as canary run labels Apr 10, 2024

Taragolis force-pushed the improve-warning-system branch from 47322f1 to 35b4fd4 Compare April 10, 2024 20:18

Taragolis mentioned this pull request Apr 10, 2024

Fix built query with implicit cartessian product in AirflowSecurityManagerV2 #38913

Merged

Taragolis added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Apr 10, 2024

Taragolis marked this pull request as draft April 11, 2024 15:38

Taragolis force-pushed the improve-warning-system branch 8 times, most recently from 5f96fee to 91cae11 Compare April 13, 2024 10:05

Taragolis removed the full tests needed We need to run full set of tests for this PR to merge label Apr 13, 2024

Taragolis force-pushed the improve-warning-system branch from 91cae11 to f81e749 Compare April 13, 2024 10:40

Taragolis marked this pull request as ready for review April 13, 2024 11:51

Taragolis requested review from ashb, jedcunningham and kaxil as code owners April 13, 2024 11:51

Taragolis requested review from vincbeck and potiuk April 13, 2024 11:51

Taragolis added full tests needed We need to run full set of tests for this PR to merge all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs labels Apr 14, 2024

Change capture warnings output format to the JSON

07fd917

Make it also work in xdist worker and exchange data with controller Provide simple script for summarize warnings with simple report

Taragolis force-pushed the improve-warning-system branch from f81e749 to 07fd917 Compare April 14, 2024 16:13

potiuk approved these changes Apr 14, 2024

View reviewed changes

vincbeck reviewed Apr 15, 2024

View reviewed changes

Apply suggestions from code review

4d9a7f9

Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>

vincbeck approved these changes Apr 15, 2024

View reviewed changes

Taragolis merged commit 1aa25f1 into main Apr 15, 2024
92 checks passed

Taragolis deleted the improve-warning-system branch April 15, 2024 17:26

potiuk added this to the Airflow 2.9.1 milestone Apr 16, 2024

Taragolis mentioned this pull request Apr 17, 2024

Append/update builtins deprecations warnings instead of prepend #39089

Merged

Taragolis mentioned this pull request Apr 25, 2024

Capture warning during setup and collect tests cases #39250

Merged

jedcunningham removed this from the Airflow 2.9.1 milestone Apr 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change capture warnings output format to the JSON #38905

Change capture warnings output format to the JSON #38905

Taragolis commented Apr 10, 2024 •

edited

Loading

potiuk left a comment

Taragolis commented Apr 10, 2024

Taragolis commented Apr 10, 2024

Taragolis commented Apr 11, 2024

Taragolis commented Apr 15, 2024

Change capture warnings output format to the JSON #38905

Change capture warnings output format to the JSON #38905

Conversation

Taragolis commented Apr 10, 2024 • edited Loading

potiuk left a comment

Choose a reason for hiding this comment

Taragolis commented Apr 10, 2024

Taragolis commented Apr 10, 2024

Taragolis commented Apr 11, 2024

Taragolis commented Apr 15, 2024

Taragolis commented Apr 10, 2024 •

edited

Loading