Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] [3006.0] Regression: Include sls that includes jinja macros leads to Rendering SLS error #64111

Closed
2 tasks done
babs opened this issue Apr 19, 2023 · 13 comments · Fixed by #64113
Closed
2 tasks done

[BUG] [3006.0] Regression: Include sls that includes jinja macros leads to Rendering SLS error #64111

babs opened this issue Apr 19, 2023 · 13 comments · Fixed by #64113
Assignees
@s0undt3ch
Labels
Bug broken, incorrect, or confusing behavior Regression The issue is a bug that breaks functionality known to work in previous releases. Sulfur v3006.1
Milestone
Sulfur v3006.1

Comments

@babs
Copy link
Contributor

babs commented Apr 19, 2023 •
edited
Loading

Description

Include of sls that includes jinja macros leads to Rendering SLS 'base:common.file1' failed: Jinja error: 'NoneType' object has no attribute 'done'
Regression introduced in 3006.0, working fine on 3005.1 and previous.

state.apply common gives the following error while state.apply common.file1 works fine.

local:
    Data failed to compile:
----------
    Rendering SLS 'base:common.file1' failed: Jinja error: 'NoneType' object has no attribute 'done'
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 471, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 185, in get_source
    self.check_cache(_template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 138, in check_cache
    ret = self.cache_file(template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 131, in cache_file
    return fcl.get_file(saltpath, "", True, self.saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1167, in get_file
    hash_server, stat_server = self.hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1414, in hash_and_stat_file
    hash_result = self.hash_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1407, in hash_file
    return self.__hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1399, in __hash_and_stat_file
    return self.channel.send(load)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/ioloop.py", line 457, in run_sync
    if not future_cell[0].done():
AttributeError: 'NoneType' object has no attribute 'done'

; line 1

---
{% from 'common/macros.jinja' import a_jinja_macro with context %}    <======================

a state id:
  cmd.run:
    - name: echo {{ a_jinja_macro("hello world") }}

[...]
---

Setup

One master and one minion running 3006.

  • VM (vmware)
  • onedir packaging

Steps to Reproduce the behavior

On the master side in a common folder:

  • macros.jinja 
    {% macro a_jinja_macro(arg) -%}
{{ arg }}
{%- endmacro %}
  • init.sls 
    include:
- common.file1
  • file1.sls 
    {% from 'common/macros.jinja' import a_jinja_macro with context %}

a state id:
  cmd.run:
    - name: echo {{ a_jinja_macro("hello world") }}

Expected behavior

On the minion side as from the master side, being able to call state.apply common without issue (as it was on 3005.1 and previous)

local:
----------
          ID: a state id
    Function: cmd.run
        Name: echo hello world
      Result: True
     Comment: Command "echo hello world" run
     Started: 15:58:45.411296
    Duration: 5.243 ms
     Changes:   
              ----------
              pid:
                  1259842
              retcode:
                  0
              stderr:
              stdout:
                  hello world

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:   5.243 ms

Versions Report

salt --versions-report 
Salt Version:
          Salt: 3006.0
 
Python Version:
        Python: 3.10.11 (main, Apr 14 2023, 05:57:16) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
        relenv: 0.11.2
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-21-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye
@babs babs added Bug broken, incorrect, or confusing behavior needs-triage labels Apr 19, 2023
@jbouter
Copy link
Contributor

jbouter commented Apr 19, 2023

Affecting us too, following code:

AttributeError: 'NoneType' object has no attribute 'done'

; line 1

---
{% from "warpnet/map.jinja" import warpnet with context %}    <======================

/etc/profile.d:
  file.directory:
    - name: /etc/profile.d
    - mode: '0755'
[...]

@jbouter jbouter mentioned this issue Apr 19, 2023
Closed
@hurzhurz
Copy link
Contributor

I think this change in the salt/utils/templates.py is causing it:
e29485e#diff-f77e29083ee4c1500d638fc2771ce95f3c76f5bf4172fa3cab7f57ea06f50318

If I manually replace the file with the version from before (https://github.com/saltstack/salt/blob/4ecfd3d3d549e655027a36746f9cf3bc9e11c6e9/salt/utils/templates.py), it seems to work.

@s0undt3ch s0undt3ch mentioned this issue Apr 19, 2023
Merged
@anilsil anilsil added this to the Sulfur v3006.1 milestone Apr 19, 2023
@dwoz dwoz removed the needs-triage label Apr 19, 2023
@dwoz
Copy link
Contributor

dwoz commented Apr 19, 2023

I am able to consistently reproduce this using the states in @babs's issue and state.apply common

@zbukhari-apex
Copy link

zbukhari-apex commented Apr 20, 2023 •
edited
Loading

I'm running into this too but with yaml_import. Pretty much everywhere I use that, this is failing.

Surprisingly it seems to be specific to client side, I have a multi-master setup and the other minions are on 3003.2, everything is just fine there and all minions are running 3003.2 as well.

One master and one minion are on 3006.0 (latest of the repo fetched April 19th 2023).

master - 3006 aka mas6
minion - 3006 aka min6
master - 3003.2 aka mas3
minion - 3003.2 aka min3

Here's a test run of a formula/recipe/etc from min3 to mas6 (success):

username@min3:~$ sudo salt-call --master=mas6 state.apply formulas.salt -l warning --state-output=changes
local:
  Name: /usr/local/bin/salt-restart.sh - Function: file.managed - Result: Clean Started: - 01:26:05.543681 Duration: 184.929 ms
  Name: /etc/salt/minion - Function: file.managed - Result: Clean Started: - 01:26:05.728844 Duration: 153.434 ms
  Name: /etc/salt/minion.d/masters.conf - Function: file.absent - Result: Clean Started: - 01:26:05.882470 Duration: 0.901 ms
  Name: /etc/salt/minion.d/master.conf - Function: file.absent - Result: Clean Started: - 01:26:05.883528 Duration: 0.812 ms
  Name: echo '*/3 * * * * root /usr/local/bin/apex-salt-restart.sh minion && rm /etc/cron.d/salt-minion-restart' > /etc/cron.d/apex-salt-minion-restart
chmod 444 /etc/cron.d/salt-minion-restart
 - Function: cmd.run - Result: Clean Started: - 01:26:05.885512 Duration: 0.005 ms
  Name: /usr/local/bin/salt-upgrader.sh - Function: file.managed - Result: Clean Started: - 01:26:05.885586 Duration: 212.082 ms
  Name: /etc/cron.d/salt-upgrader - Function: file.absent - Result: Clean Started: - 01:26:06.098022 Duration: 1.312 ms

Summary for local
------------
Succeeded: 7
Failed:    0
------------
Total states run:     7
Total run time: 553.475 ms

Here's a run of min6 to mas6 (fails):

user@min6:~$ sudo salt-call --master=mas6 state.apply formulas.salt -l warning --state-output=changes
[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 471, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 4, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 185, in get_source
    self.check_cache(_template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 138, in check_cache
    ret = self.cache_file(template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 131, in cache_file
    return fcl.get_file(saltpath, "", True, self.saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1167, in get_file
    hash_server, stat_server = self.hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1414, in hash_and_stat_file
    hash_result = self.hash_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1407, in hash_file
    return self.__hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1399, in __hash_and_stat_file
    return self.channel.send(load)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/ioloop.py", line 457, in run_sync
    if not future_cell[0].done():
AttributeError: 'NoneType' object has no attribute 'done'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 218, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 515, in render_jinja_tmpl
    raise SaltRenderError(
salt.exceptions.SaltRenderError: Jinja error: 'NoneType' object has no attribute 'done'
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 471, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 4, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 185, in get_source
    self.check_cache(_template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 138, in check_cache
    ret = self.cache_file(template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 131, in cache_file
    return fcl.get_file(saltpath, "", True, self.saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1167, in get_file
    hash_server, stat_server = self.hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1414, in hash_and_stat_file
    hash_result = self.hash_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1407, in hash_file
    return self.__hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1399, in __hash_and_stat_file
    return self.channel.send(load)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/ioloop.py", line 457, in run_sync
    if not future_cell[0].done():
AttributeError: 'NoneType' object has no attribute 'done'

; line 4

---
---
{#- # Use lsb_release -c -s to get the short oscodename or grains.oscodename
    # as we test against to determine if we need to update / upgrade. #}
{%- import_yaml "formulas/salt/data.yaml" as data %}    <======================
{#- # We're also going to put the logic for saltversion here as I wanted
    # to simplify the YAML and reduce redundancy yet make it more detailed
    # as we know have redundancy and test hosts. #}
{#- # First we account for the actual masters. #}
{%- if grains.id in data['lower']['masters'].keys() %}
[...]
---
[CRITICAL] Rendering SLS 'base:formulas.salt.upgrader' failed: Jinja error: 'NoneType' object has no attribute 'done'
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 471, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 4, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 185, in get_source
    self.check_cache(_template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 138, in check_cache
    ret = self.cache_file(template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 131, in cache_file
    return fcl.get_file(saltpath, "", True, self.saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1167, in get_file
    hash_server, stat_server = self.hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1414, in hash_and_stat_file
    hash_result = self.hash_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1407, in hash_file
    return self.__hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1399, in __hash_and_stat_file
    return self.channel.send(load)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/ioloop.py", line 457, in run_sync
    if not future_cell[0].done():
AttributeError: 'NoneType' object has no attribute 'done'

; line 4

---
---
{#- # Use lsb_release -c -s to get the short oscodename or grains.oscodename
    # as we test against to determine if we need to update / upgrade. #}
{%- import_yaml "formulas/salt/data.yaml" as data %}    <======================
{#- # We're also going to put the logic for saltversion here as I wanted
    # to simplify the YAML and reduce redundancy yet make it more detailed
    # as we know have redundancy and test hosts. #}
{#- # First we account for the actual masters. #}
{%- if grains.id in data['lower']['masters'].keys() %}
[...]
---
local:
    Data failed to compile:
----------
    Rendering SLS 'base:formulas.salt.upgrader' failed: Jinja error: 'NoneType' object has no attribute 'done'
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/templates.py", line 471, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 4, in top-level template code
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 185, in get_source
    self.check_cache(_template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 138, in check_cache
    ret = self.cache_file(template)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/jinja.py", line 131, in cache_file
    return fcl.get_file(saltpath, "", True, self.saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1167, in get_file
    hash_server, stat_server = self.hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1414, in hash_and_stat_file
    hash_result = self.hash_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1407, in hash_file
    return self.__hash_and_stat_file(path, saltenv)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/fileclient.py", line 1399, in __hash_and_stat_file
    return self.channel.send(load)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/tornado/ioloop.py", line 457, in run_sync
    if not future_cell[0].done():
AttributeError: 'NoneType' object has no attribute 'done'

; line 4

---
---
{#- # Use lsb_release -c -s to get the short oscodename or grains.oscodename
    # as we test against to determine if we need to update / upgrade. #}
{%- import_yaml "formulas/salt/data.yaml" as data %}    <======================
{#- # We're also going to put the logic for saltversion here as I wanted
    # to simplify the YAML and reduce redundancy yet make it more detailed
    # as we know have redundancy and test hosts. #}
{#- # First we account for the actual masters. #}
{%- if grains.id in data['lower']['masters'].keys() %}
[...]
---

Anyway - this is not an issue unless you think it is. This is what happens when a 3005/3006 minion tries to talk to any 3003.2 master (not following the best practice of upgrading the master first so I think this is expected, but meh):

2023-04-15 23:59:48,913 [salt.minion      :1146][ERROR   ][362612] Error while bringing up minion for multi-master. Is master at mas3 responding?
2023-04-15 23:59:58,934 [salt.crypt       :788 ][ERROR   ][362612] Sign-in attempt failed: {'enc': 'pub', 'pub_key': '_da_pub_key_', 'publish_port': 4505, 'token': '_da_token_', 'aes' : '_da_aes_', 'sig': "_da_sig_"}

@OrangeDog OrangeDog added the Regression The issue is a bug that breaks functionality known to work in previous releases. label Apr 20, 2023
@hkbakke
Copy link

hkbakke commented Apr 20, 2023 •
edited
Loading

Even the official salt-formula breaks completely with 3006.0 with this error. Although mistakes can happen, it is surprising that an issue as showstoppingly grave as this one manages to go undetected in a LTS release. It is almost like this release has never been tested on an actual production-like system before release. Also, the upgrade was a mess as the salt-master now runs as a normal user that has no access to lots of stuff like ssh keys with correct permissions for root etc,. This is good for security and I applaud it, however this should be clearly noted in the release notes so we have some pointers to why everything breaks.

Hopefully a quick fix release can happen for this specific issue so 3006 is at least usable with actual formulas.

@jbouter
Copy link
Contributor

jbouter commented Apr 20, 2023

@hkbakke Even though it is off-topic, I couldn't agree more with the salt-master process now running as a salt user. I too applaud the increase in security, but it rendered part of our environment unreachable as the master process could no longer read our stack repository, resulting in configured firewall rules being removed... I honestly wish it would have been more clearly documented as it is a possibly breaking change.

As for not being tested, it's really surprising indeed. It could have to do with the commit that causes this bug being only two weeks old, but it should have been caught. I'm all for updating software as quickly as feasible within an organisation, but it doesn't seem like salt has had a good .0 release in a long while.

@dwoz dwoz mentioned this issue Apr 20, 2023
Closed
dwoz added a commit to dwoz/salt that referenced this issue Apr 20, 2023
@dwoz
Add regression test for saltstack#64111
97b4f37 
Test importing jinja files
@tuxthepenguin84
Copy link

tuxthepenguin84 commented Apr 21, 2023 •
edited
Loading

I would ask the devs to please pull the 3006 release.

The multitude of breaking issues, lack of clear communication on the change to running as the salt user, and the fact that reverting to a previous version will still break installations (systemd file still wants to run as salt user, this tripped me up for a bit).

No mention of the salt user change here: https://docs.saltproject.io/en/latest/topics/releases/3006.html or https://saltproject.io/salt-3006-is-here/

If anything this should have be an option and not the default for the 3006 release.

s0undt3ch pushed a commit to s0undt3ch/salt that referenced this issue Apr 24, 2023
@dwoz @s0undt3ch
Add regression test for saltstack#64111
256fa7b 
Test importing jinja files
@zbukhari-apex
Copy link

zbukhari-apex commented Apr 24, 2023 •
edited
Loading

I agree and feel that 3006.0 should just be pulled.

Just the includes in and of itself is the main reason. Luckily because salt can't self-manage itself that well and I didn't put too much effort into it, the salt user change wasn't a big hit and was well done. But now I have to go back to 3005.1 in the meantime because of the include and have to play clean-up with the salt user change - luckily on only one host as I have four.

Most may not have that option because, lets face it, for the most part salt just works :-D

@defanator defanator mentioned this issue Apr 25, 2023
Closed
2 tasks
@mdinsmore
Copy link

We've also encountered the AttributeError: 'NoneType' object has no attribute 'done' error with include files on 3006.0 - reverting to the https://repo.saltproject.io/salt/py3/ubuntu/22.04/amd64/3005 repo sorted the issue.

Another issue with 3006.0 is where a dict is passed in the context to a jinja templated file (in the example below z_meta is passed in the context):

{%- set zsys = pillar.get('zsys', {}) -%}
{%- set z_url = zsys.get('url') -%}
{%- if z_meta|length %}
{%- set z_url = z_meta.get("z_url", z_url) %} <======================
{%- endif -%}

This raises the exception:
raise SaltRenderError("Jinja variable {}{}".format(exc, out), line, tmplstr)
salt.exceptions.SaltRenderError: Jinja variable 'str object' has no attribute 'get'; line 6

If the dict is read by putting {%- set z_meta = pillar.get('z_meta', {}) %} above the if statement in the jinja template, it works. I've not found anything in the release notes to say that this behaviour has been changed so assume it's a regression.

@dwoz dwoz mentioned this issue Apr 25, 2023
Closed
s0undt3ch pushed a commit to s0undt3ch/salt that referenced this issue Apr 26, 2023
@dwoz @s0undt3ch
Add regression test for saltstack#64111
11968b8 
Test importing jinja files
@bensteinberg bensteinberg mentioned this issue Apr 26, 2023
Closed
@Ninpo
Copy link

Ninpo commented Apr 27, 2023

This bit me just now with the salt-users formula, as per other reports downgrading to 3005.1 restored normal behaviour.

@s0undt3ch s0undt3ch assigned s0undt3ch and unassigned dwoz Apr 27, 2023
@s0undt3ch s0undt3ch linked a pull request Apr 27, 2023 that will close this issue
[3006.x] Make sure the file client is destroyed upon used #64113
Merged
s0undt3ch pushed a commit that referenced this issue Apr 28, 2023
@dwoz @s0undt3ch
Add regression test for #64111
3962fcd 
Test importing jinja files
@Kaelnor
Copy link

Kaelnor commented Apr 28, 2023

Same problem here after a fresh 3006 install.

The error happens in the first line of a state file importing a template. With or without context gives the same error.
Other state files are also affected in the same way.

    Data failed to compile:
----------
    Rendering SLS 'base:packages.telegraf.agent-windows' failed: Jinja error: 'NoneType' object has no attribute 'done'
Traceback (most recent call last):
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\utils\templates.py", line 471, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\jinja2\environment.py", line 1301, in render
    self.environment.handle_exception()
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\jinja2\environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\utils\jinja.py", line 185, in get_source
    self.check_cache(_template)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\utils\jinja.py", line 138, in check_cache
    ret = self.cache_file(template)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\utils\jinja.py", line 131, in cache_file
    return fcl.get_file(saltpath, "", True, self.saltenv)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\fileclient.py", line 1173, in get_file
    hash_server = self.hash_file(path, saltenv)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\fileclient.py", line 1407, in hash_file
    return self.__hash_and_stat_file(path, saltenv)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\fileclient.py", line 1399, in __hash_and_stat_file
    return self.channel.send(load)
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\utils\asynchronous.py", line 125, in wrap
    raise exc_info[1].with_traceback(exc_info[2])
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\utils\asynchronous.py", line 131, in _target
    result = io_loop.run_sync(lambda: getattr(self.obj, key)(*args, **kwargs))
  File "C:\Program Files\Salt Project\Salt\lib\site-packages\salt\ext\tornado\ioloop.py", line 457, in run_sync
    if not future_cell[0].done():
AttributeError: 'NoneType' object has no attribute 'done'

; line 1

---
{% from "packages/telegraf/map.jinja" import agent %}    <======================
{%- if agent.get('enabled', False) %}

@s0undt3ch
Copy link
Collaborator

This issue has been fixed and will be available in the upcoming 3006.1 release.

@s0undt3ch s0undt3ch mentioned this issue May 5, 2023
Closed
@darkpixel darkpixel mentioned this issue Jun 8, 2023
Closed
@cmcmarrow
Copy link
Contributor

cmcmarrow commented Oct 28, 2023 •
edited
Loading

I ran the example said example on 3006.x and state.apply common works fine now on my box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@s0undt3ch s0undt3ch

Labels
Bug broken, incorrect, or confusing behavior Regression The issue is a bug that breaks functionality known to work in previous releases. Sulfur v3006.1
Projects
None yet
Milestone
Sulfur v3006.1
Development

Successfully merging a pull request may close this issue.

[3006.x] Make sure the file client is destroyed upon used s0undt3ch/salt