Skip to content

Conversation

@ephraimbuddy
Copy link
Contributor

Older TIs won't have the dag_version_id, so we shouldn't require it in workload's TI model.

closes: #55713

@boring-cyborg boring-cyborg bot added the area:Executors-core LocalExecutor & SequentialExecutor label Sep 16, 2025
@ephraimbuddy ephraimbuddy added this to the Airflow 3.1.0 milestone Sep 16, 2025
Copy link
Contributor

@o-nikolas o-nikolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like API spec needs some updating, also should we have some tests added or at least updated to test this edgecase?

Older TIs won't have the dag_version_id, so we shouldn't require it
in workload's TI model.
@ephraimbuddy ephraimbuddy force-pushed the nullify-dagversion-in-ti-workloads branch from c14b250 to fac4f4a Compare September 16, 2025 21:54
@ephraimbuddy
Copy link
Contributor Author

Looks like API spec needs some updating, also should we have some tests added or at least updated to test this edgecase?

Thanks, updated

"""Schema for TaskInstance with minimal required fields needed for Executors and Task SDK."""

id: uuid.UUID
dag_version_id: uuid.UUID
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we execute a TI without a dag version? For it to be executed, doesn't it have to still exist, and this get reparsed to include a dag version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can execute a TI without a dag_version since the RuntimeTaskInstance model doesn't include the dag_version_id:

class RuntimeTaskInstance(TaskInstance):
. I hope my understanding is correct? Main purpose of the dag_version_id is for display not for execution

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that dag_version_id the one that the (git, etc) dag bundle backend is told to checkout?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah no, bundle_info is what the dag bundle info is.

I still wonder how it's actually possible to get to an executor with no dag_version set?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. dag bundle backend checks out dag bundle version:

version = Column(String(200), nullable=True)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it's more about the dagrun than task instance

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still -- aren't "all" DagRuns in airflow 3 meant to have a version? What is the case by which we end up without a dag version but the task in the executor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will experiment with this by reproducing this with a running TI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative to this that's less disruptive: #55884

@kaxil kaxil modified the milestones: Airflow 3.1.0, Airflow 3.1.1 Sep 19, 2025
@kaxil kaxil removed this from the Airflow 3.1.1 milestone Oct 21, 2025
@ephraimbuddy
Copy link
Contributor Author

Closing as the alternative has been merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Executors-core LocalExecutor & SequentialExecutor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Triggerer crash when migrating Airflow 2 to 3 with async dagrun already in deferred state

7 participants