Support for Jinja

[simonknittel]

I've got several projects in which my docker-compose.yml files are using the Jinja (.j2) template file format. This way I can use Ansible's template module to populate this docker-compose.yml.j2 template with variables.

However, this seems not to be supported by Renovate.

I tried extending the fileMatch for docker-compose with the following regex:

"docker-compose": {
  "fileMatch": [
    "^compose\\.ya?ml(?:\\.j2)?$"
  ]
}

This config successfully detects files named compose.yml but not compose.yml.j2.

Is there a configuration issue on my side or is this not supported by Renovate yet?

[secustor]

This will not work with the standard docker-compose manager as the files are parsed using an off the shelf YAML parser.
You could use the regex manager.

Regarding a general Jinja support. IMO not in the near future as there are too many edge cases to support it.

[guillaumededrie]

Here is an updated version that work with Renovate "^39.86.0":

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "customManagers": [
    {
      "customType": "regex",
      "fileMatch": "(^|/)(?:docker-)?compose[^/]*\\.ya?ml?\\.j2$",
      "matchStrings": [
        "(?:image: \\\"?)(?<depName>.*?)(?::(?<currentValue>.*?))?(?:@(?<currentDigest>sha256:[a-f0-9]+))?\\\"?\\n"
      ],
      "datasourceTemplate": "docker",
      "versioningTemplate": "docker"
    }
  ]
}

[secustor]

This shouldn't be necessary anymore see #18470 (reply in thread)

[guillaumededrie]

@secustor, I tried using this docker-compose manager configuration:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:best-practices"
  ],
  "docker-compose": {
    "enabled": true,
    "fileMatch": [
      "(^|/)(?:docker-)?compose[^/]*\\.ya?ml(\\.j2)?$"
    ]
  }
}

And I got this from Renovate logs:

DEBUG: docker-compose.extractPackageFile(<REDACTED>/docker-compose.yml.j2)
DEBUG: Parsing Docker Compose config YAML failed
{
  "err": {
    "message": "Schema error",
    "stack": "ZodError: Schema error\n    at Object.get error [as error] (/usr/local/renovate/node_modules/.pnpm/zod@3.24.1/node_modules/zod/lib/types.js:55:31)\n    at ZodUnion.parse (/usr/local/renovate/node_modules/.pnpm/zod@3.24.1/node_modules/zod/lib/types.js:131:22)\n    at parseSingleYaml (/usr/local/renovate/lib/util/yaml.ts:136:17)\n    at Object.extractPackageFile (/usr/local/renovate/lib/modules/manager/docker-compose/extract.ts:39:29)\n    at extractPackageFile (/usr/local/renovate/lib/modules/manager/index.ts:75:9)\n    at getManagerPackageFiles (/usr/local/renovate/lib/workers/repository/extract/manager-files.ts:58:43)\n    at /usr/local/renovate/lib/workers/repository/extract/index.ts:57:28\n    at async Promise.all (index 1)\n    at extractAllDependencies (/usr/local/renovate/lib/workers/repository/extract/index.ts:54:26)\n    at extract (/usr/local/renovate/lib/workers/repository/process/extract-update.ts:159:28)\n    at extractDependencies (/usr/local/renovate/lib/workers/repository/process/index.ts:158:26)\n    at Object.renovateRepository (/usr/local/renovate/lib/workers/repository/index.ts:71:9)\n    at attributes.repository (/usr/local/renovate/lib/workers/global/index.ts:206:11)\n    at start (/usr/local/renovate/lib/workers/global/index.ts:191:7)\n    at /usr/local/renovate/lib/renovate.ts:19:22",
    "issues": [
      "Expected object, received null",
      "Expected object, received null"
    ]
  }
  "packageFile": "<REDACTED>/docker-compose.yml.j2"
}

Here is an extract from my docker-compose.yml.j2 file related to the parsing error:

---
# {{ ansible_managed }}
version: "3.8"

networks:
  {{ network_name }}:
    external: true

services:
  app:
    image: "<NAME>@<HASH>"
    networks:
      - {{ network_name }}
    command: command
    deploy:
      mode: global
      placement:
        {{ placement | ansible.builtin.to_nice_yaml | indent(10) }}
      labels:
        - "name=value"

I suspect that the top-level network part is not well handle by the yaml parser, as I'm using a Jinja2 variable as key…

What do you think?
Regards.

[guillaumededrie]

I think that's related to parseYaml used by docker-compose.extractPackageFile which is using the massageContent that remove some Jinja2 parts (like the {{ network_name }}: or maybe the {{ placement }} one).

So, we ended up with something like this before parsing the document as yaml:

 ---
- # {{ ansible_managed }}
+ # 
 version: "3.8"

 networks:
-  {{ network_name }}:
+  
    external: true

 services:
   app:
     image: "<NAME>@<HASH>"
     networks:
-       - {{ network_name }}
+       - 
     command: command
     deploy:
       mode: global
       placement:
-         {{ placement | ansible.builtin.to_nice_yaml | indent(10) }}
+        
       labels:
         - "name=value"

I suspect that's the reason why we get the Schema error here.

[guillaumededrie]

I've opened a new PR related to this issue #33361.

[onedr0p]

@simonknittel

I was able to solve this by changing the default jinja block delimiters from {% ... %} to #% ... %# which renovate will ignore because it's treated as a YAML comment. This solution will only work if the tool you use supports changing these, I checked and the Ansible template module does support changing these. It looks a little jank but works great for my use-case.

[onedr0p]

Looks like that can be updated to support the default jinja block delimiters of {% ... %}

[onedr0p]

@secustor should I open a issue for adding two more lines to that or just try to PR it myself? 😄

[secustor]

Only collaborators can create issues, tough feel free open a PR for this.

[onedr0p]

PR opened #26682