Allow specifying a loader for the Jinja2 Environment

[ikbenale]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Allow specifying a loader when initializing the Jinja2 SandboxEnvironment to permit loading templates. See https://jinja.palletsprojects.com/en/2.10.x/api/#loaders

Motivation

If you have different prompts that share common elements, currently it isn't possible to take advantage of jinja2's include (https://jinja.palletsprojects.com/en/3.0.x/templates/#include), because there's no way to reference existing templates using the jinja2 formatter for Prompts. This is due to no loader is specified when initializing the SandboxEnvironment (here https://github.com/langchain-ai/langchain/blob/ff0df5e/libs/core/langchain_core/prompts/string.py#L59).

Proposal (If applicable)

Allow specifying a loader when initializing the jinja2 environment. Alternatively, it could be more flexible to allow the caller to supply their own jinja2 Environment object, so they can override other initialization parameters (like variable_start_string, variable_end_string, etc).

[quimey]

Hey! I'm willing to contribute with code for this idea (in particular supporting providing a custom Environment).

This will not only allow users to load templates but also use custom filters and other advanced features.

I'm looking for a guideline from a langchain maintainer on how to approach the change. A couple of ideas that come to my mind are:

• Add a "environment" attribute to the whole call chain (the PromptTemplate class and the jinja2_formatter function) that passes along the environmnet
• Have a env var or some global object with such environment and use it in the jinja2_formatter function if available.

I'm not thrilled about either, hence the advise request.

In the meantime a solution is to monkeypatch langchain to inject the correct environment in the "jinja2_formatter" function.