Investigate CI failure of `test_serde_serialize_recursion_limit`

[nailo2c]

Apache Airflow version

main (development)

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I added a new unit test, test_serde_serialize_recursion_limit, in PR #51419

def test_serde_serialize_recursion_limit(self):
    depth = sys.getrecursionlimit() - 1
    with pytest.raises(RecursionError, match="maximum recursion depth reached for serialization"):
        serialize(object(), depth=depth)

It passed the CI checks in the PR but failed after being merged into the main branch, so I removed it in PR #51737

Some error logs:
https://github.com/apache/airflow/actions/runs/15655403802/job/44105937266
https://github.com/apache/airflow/actions/runs/15653524051/job/44101637421

This unit test is for the following code snippet:

airflow/airflow-core/src/airflow/serialization/serde.py

Lines 88 to 115 in 323076a

	def serialize(o: object, depth: int = 0) -> U | None:
	"""
	Serialize an object into a representation consisting only built-in types.
	Primitives (int, float, bool, str) are returned as-is. Built-in collections
	are iterated over, where it is assumed that keys in a dict can be represented
	as str.
	Values that are not of a built-in type are serialized if a serializer is
	found for them. The order in which serializers are used is
	1. A ``serialize`` function provided by the object.
	2. A registered serializer in the namespace of ``airflow.serialization.serializers``
	3. Annotations from attr or dataclass.
	Limitations: attr and dataclass objects can lose type information for nested objects
	as they do not store this when calling ``asdict``. This means that at deserialization values
	will be deserialized as a dict as opposed to reinstating the object. Provide
	your own serializer to work around this.
	:param o: The object to serialize.
	:param depth: Private tracker for nested serialization.
	:raise TypeError: A serializer cannot be found.
	:raise RecursionError: The object is too nested for the function to handle.
	:return: A representation of ``o`` that consists of only built-in types.
	"""
	if depth == MAX_RECURSION_DEPTH:
	raise RecursionError("maximum recursion depth reached for serialization")

What you think should happen instead?

It should be works, but I'm not sure why it failed.

How to reproduce

Add test function test_serde_serialize_recursion_limit to file airflow-core/tests/unit/serialization/serializers/test_serializers.py.

Run breeze testing core-tests --use-xdist --skip-db-tests --no-db-cleanup --backend none, then you'll see

FAILED airflow-core/tests/unit/serialization/serializers/test_serializers.py::TestSerializers::test_serde_serialize_recursion_limit - TypeError: cannot serialize object of type <class 'object'>
FAILED airflow-core/tests/unit/always/test_project_structure.py::TestProjectStructure::test_providers_modules_should_have_tests - Failed: Timeout >60.0s
= 2 failed, 1652 passed, 4986 skipped, 1 xfailed, 9 warnings in 463.86s (0:07:43) =

Operating System

macos

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.