Cleared tasks get literal 'DagRunState.QUEUED' instead of the value 'queued'

[tothandor]

Apache Airflow version

2.1.3 (latest released)

Operating System

CentOS Stream release 8

Versions of Apache Airflow Providers

None of them are relevant

Deployment

Virtualenv installation

Deployment details

mkdir /srv/airflow
cd /srv/airflow
virtualenv venv
source venv/bin/activate
pip install apache-airflow==2.1.3
AIRFLOW_HOME and AIRFLOW_CONFIG is specified via environment variables in /etc/sysconfig/airflow, which is in turn used as EnvironmentFile in systemd service files.
systemctl start airflow-{scheduler,webserver,kerberos}
Python version: 3.9.2
LocalExecutors are used

What happened

On the Web UI, I had cleared failed tasks, which have been cleared properly, but the DagRun became black with a literal value of "DagRunState.QUEUED", therefore it can't be scheduled again.

What you expected to happen

DagRun state should be 'queued'.

How to reproduce

Just clear any tasks on the Web UI. I wonder how could it be that nobody noticed this issue.

Anything else

Here's a patch to fix it. Maybe the str method should be different, or the database/persistence layer should handle this, but for now, this solves the problem.

--- airflow/models/dag.py.orig  2021-08-28 09:48:05.465542450 +0200
+++ airflow/models/dag.py       2021-08-28 09:47:34.272639960 +0200
@@ -1153,7 +1153,7 @@
         confirm_prompt=False,
         include_subdags=True,
         include_parentdag=True,
-        dag_run_state: DagRunState = DagRunState.QUEUED,
+        dag_run_state: DagRunState = DagRunState.QUEUED.value,
         dry_run=False,
         session=None,
         get_tis=False,
@@ -1369,7 +1369,7 @@
         confirm_prompt=False,
         include_subdags=True,
         include_parentdag=False,
-        dag_run_state=DagRunState.QUEUED,
+        dag_run_state=DagRunState.QUEUED.value,
         dry_run=False,
     ):

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[uranusjr]

The proposed solution is wrong. The .value call should happen in Jinja templates or www/views.py (for presentation), not in models.

[tothandor]

This has nothing to do with presentation. The appropriate value controls if a dagrun should be (re)executed or not.
The default value specified for clear() method's dag_run_state resulted a bad value.
See the contents of the dag_run table below.

[uranusjr]

So the problem is we’re storing wrong values to the database? The change should be done at the boundary between the db and Pythron if that’s the case. Either we call .value when passing the value to SQLAlchemy, or we should change the state field to automatically do that when it receives an Enum.

[kaxil]

I haven't been able to reproduce yet with Postgres & Python 3.8, what Metadata Database do you use Postgres, MySQL or SQLite? And which version

[kaxil]

If you are using MySQL with MySQLdb driver -- then you might see this error. I will have a patch ready in few mins

[kaxil]

#17886 Should fix it -- can you try it with that change @tothandor please

[tothandor]

It's working now. Thanks a lot! 🙏