[BUG] Include + exclude + override with a require results in "The following requisites were not found"

[gigi206]

Description
I have complex salt code that target some servers that include states and for some specific servers I need to override some states and exclude others. That's why the following code has is reduced to the simplest and might seem strange at 1st glance.

I think the bug is in the extend:test1 with a require because if you comment the require, the code works perfectly.

Steps to reproduce the behavior

• Create /srv/salt/top.sls file that called test1.sls and exclude.sls:

$ cat top.sls
base:
  '*':
    - test1
    - exclude

• Create /srv/exclude.sls that exclude test2.sls:

$ cat exclude.sls
exclude:
  - test2

• Create /srv/test1.sls that include test2.sls:

$ cat test1.sls
include:
  - test2

test1:
  cmd.run:
    - name: echo test1

• Create /srv/test2.sls that extend test1 and require test2 in the same file:

$ cat test2.sls
extend:
  test1:
    cmd.run:
      - name: echo "override test1 in test2"
      - require:
        - id: test2

test2:
  cmd.run:
    - name: echo test2

• Apply the state:

$ salt-call --local state.apply
local:
----------
          ID: test1
    Function: cmd.run
        Name: echo "override test1 in test2"
      Result: False
     Comment: The following requisites were not found:
                                 require:
                                     id: test2
     Started: 08:32:08.872336
    Duration: 0.005 ms
     Changes:

Summary for local
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:   0.005 ms

Expected behavior
The code must work. Indeed the test2.sls is excluded in exclude.sls (and exclude win agains an include) and the code failed because it search the id test2 (require test2 in test1 in the test2.sls) but this require is in test2 that is excluded in exclude.sls.

Versions Report
All versions are concerned

[OrangeDog]

This may be a case of #61121

All versions are concerned

What does this mean? What versions have you tested?
Please provide a versions report for one of them.

[gigi206]

Salt Version:
          Salt: 3004.2
 
Dependency Versions:
          cffi: 1.15.1
      cherrypy: Not Installed
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
      M2Crypto: 0.38.0
          Mako: Not Installed
       msgpack: 1.0.3
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.12.0
        pygit2: Not Installed
        Python: 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0]
  python-gnupg: Not Installed
        PyYAML: 6.0
         PyZMQ: 23.2.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: garuda Soaring Talon
        locale: utf-8
       machine: x86_64
       release: 5.18.9-zen1-1-zen
        system: Linux
       version: Garuda Linux Soaring Talon

[Wanick95]

Any news about this trouble ? We have the same problem here.

[OrangeDog]

@Wanick95 have you tested with 3005?

[Wanick95]

Yes I have test it.

Salt Version:
Salt: 3005

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.8.1
docker-py: 5.0.3
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.2
libgit2: Not Installed
M2Crypto: Not Installed
Mako: 1.1.3
msgpack: 1.0.4
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.15.0
pygit2: Not Installed
Python: 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
python-gnupg: Not Installed
PyYAML: 5.4.1
PyZMQ: 20.0.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.3

System Versions:
dist: ubuntu 22.04 Jammy Jellyfish
locale: utf-8
machine: x86_64
release: 5.15.0-47-generic
system: Linux
version: Ubuntu 22.04 Jammy Jellyfish

[Wanick95]

I think the problem is during the computing phase.
The extends are computing before exclude sls.
For example we can use this code :

• /srv/salt/top.sls

base:
  '*':
    - test1
    - exclude

• /srv/salt/exclude.sls

exclude:
  - sls: test2

• /srv/salt/test1.sls

include:
  - test2

test1:
  cmd.run:
    - name: echo test1

• /srv/salt/test2.sls

extend:
  test1:
    cmd.run:
      - name: echo "override test1 in test2"

test2-id:
  cmd.run:
    - name: echo test2

• salt-call --local state.apply

local:
----------
          ID: test1
    Function: cmd.run
        Name: echo "override test1 in test2"
      Result: True
     Comment: Command "echo "override test1 in test2"" run
     Started: 15:56:40.498377
    Duration: 9.302 ms
     Changes:
              ----------
              pid:
                  30431
              retcode:
                  0
              stderr:
              stdout:
                  override test1 in test2

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:   9.302 ms

[agraul]

At a first glance, it looks like it shouldn't be too hard to not process "extend declarations" that are specified in sls files that are excluded. In reconcile_extend(), all needed information seems to be available. This is my WIP branch that aims to fix this issue: master...agraul:fix/extend-exclude-interaction