AWS BatchOperator does not fetch log entries for deferred jobs

[delagoya]

Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

apache-airflow-providers-amazon 9.0.0

Apache Airflow version

2.10.3

Operating System

Amazon Linux 2

Deployment

Amazon (AWS) MWAA

Deployment details

Default deployment + IAM permissions for AWS Batch for the MWAA execution role.

What happened

Using the BatchOperator to run a AWS Batch job as a deferred task, the logs are not retrieved despite setting the awslogs_enabled parameter to true.

CloudWatch log stream entries for the Batch job are only able to be brought into the Airflow console when the Airflow task is not deferred. This is due to the fetch functionality only implemented in the monitor_job function, which is only called when a job is not deferred.

What you think should happen instead

At the end of a deferred job, the CloudWatch log stream entries should be fetched and brought into the console if awslogs_enabled is set to true.

How to reproduce

Run any dag that has a deferred task using the BatchOperator.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[jroachgolf84]

@delagoya, would you like this issue assigned to you?

[delagoya]

Yes working on PR.

…

On Sat, Aug 16, 2025 at 9:24 AM Jake Roach ***@***.***> wrote: *jroachgolf84* left a comment (apache/airflow#54561) <#54561 (comment)> @delagoya <https://github.com/delagoya>, would you like this issue assigned to you? — Reply to this email directly, view it on GitHub <#54561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAFRWM7TVVJPUUMA3RACL3N4WJLAVCNFSM6AAAAACEAHRQVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCOJTGY3DQNJYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[yash1thsa]

Hey is this issue still open? would like to take a shot

[delagoya]

Yes, still an issue and I have not had a chance to get at a solution yet. Feel free to create a PR.

[yash1thsa]

Thanks @delagoya , i was able to reproduce the issue. As a next step, i was trying to play around with the tests(test_batch.py). I am encountering lot of issue to set it up. Any pointers or documentation can help me fastrack

[yash1thsa]

@delagoya - please have a look at my draft PR and let meknow your thoughts - https://github.com/yash1thsa/airflow/pull/1/files#diff-50f4cf88b4a0c94b350b4b6f7ca51d6dff3faeb046ee0d2060d495bdd96ae7df