dag_run timestamp incorrect due to setting time zone

[linwanlong]

Apache Airflow version

3.0.3

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Problem description

This is the time zone in the airflow configuration, whether set to UTC or Asia/Shanghai will cause the following problems.

# Default timezone in case supplied date times are naive
# can be `UTC` (default), `system`, or any `IANA <https://www.iana.org/time-zones>`
# timezone string (e.g. Europe/Amsterdam)
#
# Variable: AIRFLOW__CORE__DEFAULT_TIMEZONE
#
default_timezone = utc

1. I set the time zone of all containers in the airflow cluster to Asia/Shanghai.

2. Manually executing the RESTful API interface triggers dag_run run.
   http://<your-airflow-host>/api/v2/dags/{dag_id}/dagRuns

3. The dag_run_id received back is manual__2025-07-31T17:34:53.759767+08:00.

4. The confusing question came, logical_date turned out to be 2025-07-31 09:34:53.597894+08

Results shown

Below is a screenshot of my postgres database:

More details

After many experiments on my own, I found some more detailed problems:

curl -X POST http://<your-airflow-host>/api/v2/dags/{dag_id}/dagRuns \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-token>" \
  -d '{
    "conf": {
    },
    "logical_date": "2025-07-31T09:34:53.759767+00:00"
  }'

If the logical_date field in the interface request body is UTC, then the following error will appear in the database.

If the logical_date field in the interface request body is Asia/Shanghai, then the following seemingly normal record will appear in the database.

Both of these situations will cause the log timestamp format to be abnormal.

Just set the time zone in the container to UTC and the above problem will not occur.

What you think should happen instead?

I think there is a problem with the program's resolution of timestamps with timezones and the logic of generating run_id, which depends on the host's timezone but does not solve the problem of timezone conversion

How to reproduce

1. Set the time zone of the platform where the airflow process is located to Asia/Shanghai.
2. Manually trigger the dag_run interface to change the logical_date to two different time zone formats: UTC and Asia/Shanghai.
3. Observe the correspondence between the returned run_id and the logical_date.
4. Observe the log timestamps of the task in the dag.

Operating System

Ubuntu 20.04.6 LTS \n \l

Versions of Apache Airflow Providers

apache-airflow-providers-fab==2.3.0

Deployment

Docker-Compose

Deployment details

I manually packaged a custom image via dockerfile, and the base image is python:3.10.13-slim

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Abhishek1559]

I am working on this.

[Abhishek1559]

While generating the run_id, the code should utilize the run type and the logical date. Currently it is derived from run type and run_after instead.

The run_after field is based on;
run_after = run_after or timezone.coerce_datetime(timezone.utcnow()).

If we don't provide a run_after field in the API call, it will take it as utcnow() and based on the same, we get to see the inconsistency.

While running manually from UI, both logical date and run_after will be same based on the utcnow(), so we are getting the desired results.

To make this consistent, we can derive the run_id from logical_date instead of the run_after field.

[ashb]

@linwanlong What database are you using please?

[Abhishek1559]

The orignal issue uses postgres and I am using SQLite to replicate the same

[Abhishek1559]

Hi @ashb ,

The code defaults run after as utcnow() if run_after is not provided in the call
(run_after = run_after or timezone.coerce_datetime(timezone.utcnow())),

so that causes the run_id to have incorrect timestamp.

The logical date is in the provided(local) timezone. However the run_after is not provided and set as utc by default in the code.

If you manually provide run_after field in the api call, this issue would not come as it will take the run_after as the provided one.
Also same happens if timezone is set to utc and run_after is not provided as in that case the default and the timezone set in the code will be the same.

To resolve the same, should we prioritize the given/local timezone setup? or should we transform everything into UTC to follow the consistent behavior as in run_after.

(One more thing observed, if you give same run_after in two api calls, it will throw the same UniqueConstraintViolation error making the current logic not reliable. I understand the possibility of having same run_after is thin, still the logic is a bit flawed. Please correct me if I am wrong.)

I the doubt is relevant, should I go for a resolution with my analysis? Please feel free to cross question and help if i misunderstood something.

Thanks

[linwanlong]

@linwanlong What database are you using please?

@ashb I'm very sorry, I just saw the message, the database is postgres version is 17.5