Skip to content

Commit

Permalink
fix: Add comment on classname overriding
Browse files Browse the repository at this point in the history
Note: we do need to specify the classname, which should be "builtins.tuple":

The serialize method (in airflow/serialization/serializers/builtin.py) does qualname() on the namedtuple, which returns "airflow.providers.databricks.hooks.Row" (the namedtuple dynamically created in the hook). If this is used as classname, it will fail to deserialize: there won't be any deserializer for it.

If we hardcode it to "builtins.tuple", it gets deserialized correctly.
  • Loading branch information
joffreybienvenu-infrabel committed Feb 17, 2024
1 parent 8cb701f commit 807845a
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion airflow/serialization/serde.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,9 @@ def serialize(o: object, depth: int = 0) -> U | None:
qn = qualname(o)
classname = None

# serialize namedtuple like tuples
# Serialize namedtuple like tuples
# We also override the classname returned by the builtin.py serializer. The classname
# has to be "builtins.tuple", so that the deserializer can deserialize the object into tuple.
if _is_namedtuple(o):
qn = "builtins.tuple"
classname = qn
Expand Down

0 comments on commit 807845a

Please sign in to comment.