Skip to content

The TestEmrCreateJobFlowOperator is flaky #58907

@potiuk

Description

@potiuk

The TestEmrCreateJobFlowOperator is flaky. Sometimes it fails with:

================================================================================================= ERRORS =================================================================================================
____________________________________________________________ ERROR at teardown of TestEmrCreateJobFlowOperator.test_render_template_from_file ____________________________________________________________
devel-common/src/tests_common/pytest_plugin.py:1965: in clean_dags_dagruns_and_dagbundles
    clear_db_dags()
devel-common/src/tests_common/test_utils/db.py:267: in clear_db_dags
    ).delete()  # todo: this should not be necessary because the fk to DagVersion should be ON DELETE SET NULL
      ^^^^^^^^
/usr/python/lib/python3.12/site-packages/sqlalchemy/orm/query.py:3222: in delete
    result = self.session.execute(
/usr/python/lib/python3.12/site-packages/sqlalchemy/orm/session.py:1720: in execute
    result = compile_state_cls.orm_setup_cursor_result(
/usr/python/lib/python3.12/site-packages/sqlalchemy/orm/persistence.py:1889: in orm_setup_cursor_result
    cls._do_post_synchronize_evaluate(session, result, update_options)
/usr/python/lib/python3.12/site-packages/sqlalchemy/orm/persistence.py:2483: in _do_post_synchronize_evaluate
    attributes.instance_state(obj)
E   AttributeError: 'NoneType' object has no attribute '_sa_instance_state'
------------------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------------------
2025-12-01T13:55:12.129141Z [info     ] Creating ORM DAG for test_dag_id [airflow.dag_processing.collection]
2025-12-01T13:55:12.158303Z [info     ] Creating job flow using aws_conn_id: aws_default, emr_conn_id: emr_default [airflow.task.operators.airflow.providers.amazon.aws.operators.emr.EmrCreateJobFlowOperator]
2025-12-01T13:55:12.164657Z [info     ] Job flow with id j-8989898989 created [airflow.task.operators.airflow.providers.amazon.aws.operators.emr.EmrCreateJobFlowOperator]
------------------------------------------------------------------------------------------ Captured stderr call ------------------------------------------------------------------------------------------
2025-12-01T13:55:12.122810Z [info     ] Sync 1 DAGs                    [airflow.serialization.serialized_objects]
------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------
INFO     airflow.serialization.serialized_objects:serialized_objects.py:2952 Sync 1 DAGs
------------------------------------------------------------------ generated xml file: /files/test_result-providers_amazon-postgres.xml ------------------------------------------------------------------

For example here: https://github.com/apache/airflow/actions/runs/19824640938/job/56795807970#step:8:2342

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions