You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Template rendering for the livy operator doesn*t work (tested with args and files), nothing is rendered in the rendered template tab in the task instance view and the un-rendered string is sent to livy, e.g.:
I verified that template rendering is working in other DAGS by running the included example_bash_operator DAG
What you think should happen instead
When downgrading the version to apache-airflow-providers-apache-livy==3.7.0, rendered template tab is filled with values and the rendered string is sent to livy:
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
Apache Airflow Provider(s)
apache-livy
Versions of Apache Airflow Providers
apache-airflow-providers-apache-livy==3.7.1
Apache Airflow version
2.8.1
Operating System
macOS Sonoma 14.0 (23A344)
Deployment
docker compose
Deployment details
No response
What happened
Template rendering for the livy operator doesn*t work (tested with
args
andfiles
), nothing is rendered in the rendered template tab in the task instance view and the un-rendered string is sent to livy, e.g.:Submitting job {"file": "/spark-examples.jar", "className": "org.apache.spark.examples.SparkPi", "args": ["{{ task.task_id }}"], "numExecutors": 1, "conf": {"spark.shuffle.compress": "false"}}
I verified that template rendering is working in other DAGS by running the included
example_bash_operator
DAGWhat you think should happen instead
When downgrading the version to
apache-airflow-providers-apache-livy==3.7.0
, rendered template tab is filled with values and the rendered string is sent to livy:Submitting job {"file": "/spark-examples.jar", "className": "org.apache.spark.examples.SparkPi", "args": ["pi_java_task"], "numExecutors": 1, "conf": {"spark.shuffle.compress": "false"}}
How to reproduce
Following the instructions from https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html using docker compose and building the following Dockerfile:
A custom requirements.txt:
Running the following example DAG:
To get template rendering working change the requirements.txt to the following and rebuild/deploy airflow:
Anything else
Seems the issue was introduced in #36490 to fix #36484
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: