Skip to content

Conversation

@Owen-CH-Leung
Copy link
Contributor

closes: #53526

As described in the issue, the existing ElasticSearchTaskHandler / OpensearchTaskHandler does not render log well. As compared to local task log rendering, we can see that local task log is able to render logs such that it's filterable by log level.

The root cause is that the handler is grouping all log lines into a single StructuredLogMessage. Instead of grouping, each log line should form its own StructuredLogMessage.

message = header + [
StructuredLogMessage(event=concat_logs(hits)) for hits in logs_by_host.values()
]

@Owen-CH-Leung Owen-CH-Leung marked this pull request as ready for review July 24, 2025 11:54
@eladkal eladkal merged commit 70dbf40 into apache:main Jul 25, 2025
71 checks passed
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 7, 2025
…3639)

* Config Elasticsearch/OpensearchTaskHandler to render log well
fweilun pushed a commit to fweilun/airflow that referenced this pull request Aug 11, 2025
…3639)

* Config Elasticsearch/OpensearchTaskHandler to render log well
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ElasticSearch / Opensearch Task Handler does not render task log well

2 participants