[CT-1332] [Bug] Removing the policy_tags property from a column does not remove the policy tag(s)

[nolan-jardine]

Is this a new bug in dbt-bigquery?

• I believe this is a new bug in dbt-bigquery
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Policy tags cannot be removed from columns by removing the policy_tags property.

Expected Behavior

Removing the policy_tags property from a column will remove the policy tags from that column that dbt initially added.

Steps To Reproduce

As an example, let's say we define the model's corresponding .yml file as follows:
With the following definition:

models:
- name: policy_tag_table
  columns:
    - name: field
      policy_tags:
        - 'need_to_know'

The model is executed and the need_to_know policy tag is added to the field column. Now if we decide that we no longer want the need_to_know policy tag attached to the field column, one would think that removing the policy_tags property would remove the policy tags (similar to the behavior of the description property).

models:
- name: policy_tag_table
  columns:
    - name: field

However, this does not remove the policy tags. Currently, to use dbt to remove the policy tags, an empty list of policy tags needs to be provided instead:

models:
- name: policy_tag_table
  columns:
    - name: field
      policy_tags:
        - 

Relevant log output

No response

Environment

- OS: macOS Monterey 12.6
- Python: 3.10.6
- dbt-core: 1.2.1
- dbt-bigquery: 1.2.0

Additional Context

No response

[dbeatty10]

Thanks for reporting this along with an excellent write-up @nolan-jardine !

Agreed with your expectation that removing the policy_tags property from a column will remove the policy tags from that column, just like the behavior of the description property. (For the curious, policy_tags was originally implemented in dbt-labs/dbt-core#2589 and has been essentially unchanged since.)

I think this is a easy modification to make, and the logic for description and policy_tags happen to be right next to each other:

dbt-bigquery/dbt/adapters/bigquery/impl.py

Lines 618 to 622 in a3d7054

	if dotted_column_name in dbt_columns:
	column_config = dbt_columns[dotted_column_name]
	bq_column_dict["description"] = column_config.get("description")
	if column_config.get("policy_tags"):
	bq_column_dict["policyTags"] = {"names": column_config.get("policy_tags")}

Without having actually tried it, I'm guessing the new logic would look something like:

            bq_column_dict["description"] = column_config.get("description")
            bq_column_dict["policyTags"] = {"names": column_config.get("policy_tags", DEFAULT_VALUE_WHEN_NULL)}

where DEFAULT_VALUE_WHEN_NULL is replaced with the applicable default value (e.g., something like "" or []) OR removed altogether if None is the applicable default. The implementer would need to suss out the applicable default value here.

Bonus points for an implementation that makes sure there are functional tests for both adding and removing descriptions and policy_tagss.

I'm going to label this as a "good_first_issue" for an interested open-source contributor to pick up.