Support a global template_searchpath config option in Airflow

[gunjisairevanth]

Description

Apache Airflow version
3

What happened?
While upgrading to Airflow 3, we restructured our DAGs to follow the recommended best practices — specifically, moving shared SQL templates out of the dags/ directory and into the include/ folder.

However, once we made this change, our DAGs could no longer find the SQL files unless we explicitly added a template_searchpath argument inside every single DAG definition, pointing to an absolute path like /usr/local/airflow/include.

This works functionally, but it's not ideal. It requires every DAG to include the same configuration snippet, which makes the code harder to maintain — especially in projects with many DAGs.

What you think should happen instead?
There should be a way to define a default template_searchpath globally for the entire Airflow deployment — either in airflow.cfg or via an environment variable like AIRFLOW__CORE__TEMPLATE_SEARCHPATH. That way, DAG authors wouldn’t need to repeat the same setting in every DAG file just to access shared templates or SQL files.

Use case/motivation

We want all our DAGs to be able to access SQL files from a common shared director. Right now, each DAG must manually set template_searchpath for this to work. Being able to define this path globally at the deployment level would reduce code duplication and simplify DAG authoring across multiple teams and projects.

Related issues

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[jroachgolf84]

@amoghrajesh, curious your thoughts on something like this. I think this would be a nice feature...

[Brunda10]

I'd like to pick up this issue if it's not already being handled and looks like a usefull feature.
Please let me know if there are any constraints, expectations, or directions I should follow before I start working on a solution.

[Brunda10]

Hi @amoghrajesh ,

As you suggested, I tried using the environment variable AIRFLOW__CORE__TEMPLATE_SEARCHPATH in the Breeze terminal, and used it in the DAGs with template_searchpath able to access it.In development mode, we can set the default path while starting Airflow.
Is it okay to set it this way , or is there any other preferred approach for production as well?

[potiuk]

yes. Adding global config like that is a good idea.

[Brunda10]

Hi @potiuk amd @amoghrajesh ,

In development (using Breeze), we can access the value if we export AIRFLOW__CORE__TEMPLATE_SEARCHPATH="" inside the Breeze terminal. That works fine. Also, i have added support for setting default path when Breeze starts — via CLI option like breeze start-airflow --template-searchpath "" to make it easier instead of exporting every time.

For Production, I tried passing the variable as an environment variable in the docker run command, and it worked — Airflow was able to read the template path inside the dag from the config.

But I’m not sure if we’re supposed to set a default path somewhere in deployment, or if it always has to be passed manually. Would be great to know what the recommended way is for setting it up .

[potiuk]

But I’m not sure if we’re supposed to set a default path somewhere in deployment, or if it always has to be passed manually. Would be great to know what the recommended way is for setting it up .

I have no idea as well - I don't even know where this behaviour you are observing is coming from, because it is not documented in config_templates/config.yml and resulting documentation - so it woudl require to go through codebase and check/find out how it is derived. But yes, I think it would be a good idea to have it as parameter likely. I would not go as far as adding a separate parameter to breeze - those things are easier to be configured via init.sh script following https://github.com/apache/airflow/blob/main/dev/breeze/doc/02_customizing.rst#customizing-breeze-startup

[Brunda10]

Thanks @potiuk, makes sense!
I’ve tested this solution by adding the line below to files/airflow-breeze-config/init.sh:

export AIRFLOW__CORE__TEMPLATE_SEARCHPATH=/files/my_templates

[Brunda10]

@potiuk I've made the change you suggested:

Introduced a new template_searchpath parameter in config_templates/config.yml to make the configuration explicit.

Investigated the codebase (especially around DAG rendering logic) to trace where the Jinja template environment is created and used.