Missing log file when there's more than one attempt

[r-magalhaes]

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

Hello,

When there's more than one attempt in a task execution, I've noticed that the first attempt log file is missing from the worker that the webserver is trying to fetch the log from.

Airflow version v2.5.3.

What you think should happen instead

No response

How to reproduce

If one of your tasks has 2 attempts for example, the first attempt log file seems to be missing from the worker.

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[hussein-awala]

Could you provide more details about your issue and your airflow deployment?

• Which executor are you using?
• Are you using remote logging?
• the first attempt log file is missing from the worker what do you get exactly?

Also, did you check if the issue was already fixed in a newer version? (for ex: #32561)

[r-magalhaes]

Hey @hussein-awala,

So we're using CeleryExecutor and we're not using Remote Logging.

In the first attempt logs we're getting this

I check inside the worker it tries to fetch the log from and the file is not there.

I think there's a plan to update Airflow's version but as for right now we need to guarantee a few things before the update.

[github-actions]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions]

This issue has been closed because it has not received response from the issue author.