[CT-231] [Bug] Macro change not picked up by state:modified+ if macro is invoked after an unchanged macro in the model code

[saraleon1]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

In dbt core version 1.0.0, if you reference an unchanged macro in model code before referencing the changed macro, the macro change will not be picked up by the state:modified+ flag. For example, if I have 2 macros:

{% macro unchanged_macro() %}
1 as column2


{% endmacro %}

and

{% macro changed_macro() %}
2 as column1


{% endmacro %}

Referenced in 2 different models like so:

WITH my_test AS (
    SELECT 
    
     {{ changed_macro() }},
     {{ unchanged_macro() }}
)

SELECT * from my_test

and the second model is:

WITH my_test AS (
    SELECT 

     {{ unchanged_macro() }},
     {{ changed_macro() }}
)

SELECT * from my_test

and then make a change and pull request against the macro 'changed macro', only the first model will run when invoked by:
dbt build --select state:modified+

This looks to be due to the order in which the macros are called in the model.

Expected Behavior

both models are identified as modified, regardless of the order in which macros are called in the model code

Steps To Reproduce

1. with a project on dbt core 1.0.0, create 2 macros:

{% macro unchanged_macro() %}
1 as column2


{% endmacro **%}

and:

{% macro changed_macro() %}
2 as column1


{% endmacro %}

2. also create 2 models that call those macros:

WITH my_test AS (
    SELECT 

     {{ unchanged_macro() }},
     {{ changed_macro() }}
)

SELECT * from my_test

and:

WITH my_test AS (
    SELECT 
    
     {{ changed_macro() }},
     {{ unchanged_macro() }}
)

SELECT * from my_test

3. merge all changes so far
4. create a CI/CD job with a state:modified+ command, like so: dbt build --select state:modified+
5. modify the macro 'changed_macro' such as adding another column like so:

{% macro changed_macro() %}
2 as column1,
3 as column3


{% endmacro %}

6. look at the run logs for the CI/CD job, and see that only the model that calls the changed_macro first is identified and run

Relevant log output

console logs:


13:36:14  [�[33mWARNING�[0m]: Deprecated functionality
The `source-paths` config has been renamed to `model-paths`. Please update your
`dbt_project.yml` configuration to reflect this change.
13:36:14  [�[33mWARNING�[0m]: Deprecated functionality
The `data-paths` config has been renamed to `seed-paths`. Please update your
`dbt_project.yml` configuration to reflect this change.
13:36:14  Running with dbt=1.0.1
13:36:14  Partial parse save file not found. Starting full parse.
13:36:15  Found 9 models, 4 tests, 0 snapshots, 1 analysis, 368 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
13:36:15  
13:36:18  Concurrency: 4 threads (target='default')
13:36:18  
13:36:18  1 of 1 START view model dbt_cloud_pr_45850_24.macro_test........................ [RUN]
13:36:19  1 of 1 OK created view model dbt_cloud_pr_45850_24.macro_test................... [�[32mSUCCESS 1�[0m in 1.21s]
13:36:19  
13:36:19  Finished running 1 view model in 4.23s.
13:36:19  
13:36:19  �[32mCompleted successfully�[0m
13:36:19  
13:36:19  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

### Environment

```markdown
dbt Cloud
- dbt: 1.0.0 latest

What database are you using dbt with?

snowflake

Additional Context

No response

[jtcohen6]

Thanks for the detailed writeup @saraleon1!

I was able to reproduce this locally. Both models show both macros in their depends_on.macros in manifest.json. After changing changed_macro, however, I see only:

$ dbt ls -s state:modified --state state/
testy.model_a

I'm pretty sure the problem lies in the recursive logic for this method (no surprise, I wrote this code):

dbt-core/core/dbt/graph/selector_methods.py

Lines 452 to 467 in f3c7b6b

	def recursively_check_macros_modified(self, node, previous_macros):
	# loop through all macros that this node depends on
	for macro_uid in node.depends_on.macros:
	# avoid infinite recursion if we've already seen this macro
	if macro_uid in previous_macros:
	continue
	previous_macros.append(macro_uid)
	# is this macro one of the modified macros?
	if macro_uid in self.modified_macros:
	return True
	# if not, and this macro depends on other macros, keep looping
	macro_node = self.manifest.macros[macro_uid]
	if len(macro_node.depends_on.macros) > 0:
	return self.recursively_check_macros_modified(macro_node, previous_macros)
	else:
	return False

Namely, if the first macro doesn't have any dependencies of its own, we return False (and break the loop) rather than continuing to loop through the initial set of node.depends_on.macros. If I change those last few lines to:

            if len(macro_node.depends_on.macros) > 0:
                return self.recursively_check_macros_modified(macro_node, previous_macros)
            elif len(node.depends_on.macros) > len(previous_macros):
                continue
            else:
                return False

This returns the correct result:

$ dbt ls -s state:modified --state state/
testy.model_a
testy.model_b

[iknox-fa]

@leahwicz This showed up in BLG today but it's already been closed, maybe check the GHA code to see why it's still open in Jira? Note-- the JIRA ticket seems to ber a duplicate (See CT-163 and CT-231)