Skip to content
This repository has been archived by the owner on Jan 16, 2023. It is now read-only.

Snapshots #1

Merged
merged 2 commits into from
Oct 5, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -164,35 +164,52 @@ vars:
| ------------- | ------------------------------------------------------------------------------ |
| sources | `dbt run-operation apply_masking_policy --args '{"resource_type": "sources"}'` |
| models | `dbt run -- model <model-name>` |
| snapshots | `dbt snapshot --select <snapshot-name> --target <target-name>` |

- Alternatively, you can also apply the masking policies by specifying below `post-hook` OR `on-run-end` to `dbt_project.yml`

**Example** : dbt_project.yml

(For models)
```yaml
models:
post-hook:
- "{{ dbt_snow_mask.apply_masking_policy('models') }}"
```

(For snapshots)
```yaml
snapshots:
post-hook:
- "{{ dbt_snow_mask.apply_masking_policy('snapshots') }}"
```

# How to remove masking policy ?

- Remove the masking policy applied by this package by running below commands

| Resource Type | Command |
| ------------- | -------------------------------------------------------------------------------- |
| sources | `dbt run-operation unapply_masking_policy --args '{"resource_type": "sources"}'` |
| models | `dbt run-operation unapply_masking_policy --args '{"resource_type": "models"}'` |
| Resource Type | Command |
| ------------- | ----------------------------------------------------------------------------------- |
| sources | `dbt run-operation unapply_masking_policy --args '{"resource_type": "sources"}'` |
| models | `dbt run-operation unapply_masking_policy --args '{"resource_type": "models"}'` |
| snapshots | `dbt run-operation unapply_masking_policy --args '{"resource_type": "snapshots "}'` |

- Alternatively, you can also apply the unmasking policies by specifying below `post-hook` OR `on-run-end` to `dbt_project.yml`

**Example** : dbt_project.yml

(For models)
```yaml
models:
post-hook:
- "{{ dbt_snow_mask.unapply_masking_policy('models') }}"
```
(For snapshots)
```yaml
snapshots:
post-hook:
- "{{ dbt_snow_mask.unapply_masking_policy('snapshots') }}"
```

# How to validate masking policy ?

2 changes: 1 addition & 1 deletion macros/snow-mask/apply-policy/apply_masking_policy.sql
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@

{% if resource_type == "sources" %}
{{ dbt_snow_mask.apply_masking_policy_list_for_sources(meta_key) }}
{% elif resource_type == "models" %}
{% elif resource_type|lower in ["models", "snapshots"] %}
{{ dbt_snow_mask.apply_masking_policy_list_for_models(meta_key) }}
{% endif %}

Original file line number Diff line number Diff line change
@@ -10,10 +10,10 @@
{% set schema = model.schema %}
{% set model_resource_type = model.resource_type | string %}

{% if model_resource_type|lower in ["model"] %}
{% if model_resource_type|lower in ["model", "snapshot"] %}

{# This dictionary stores a mapping between materializations in dbt and the objects they will generate in Snowflake #}
{% set materialization_map = {"table": "table", "view": "view", "incremental": "table"} %}
{% set materialization_map = {"table": "table", "view": "view", "incremental": "table", "snapshot": "table"} %}

{# Append custom materializations to the list of standard materializations #}
{% do materialization_map.update(fromjson(var('custom_materializations_map', '{}'))) %}
@@ -77,9 +77,9 @@
{% set schema = node.schema | string %}
{% set node_unique_id = node.unique_id | string %}
{% set node_resource_type = node.resource_type | string %}
{% set materialization_map = {"table": "table", "view": "view", "incremental": "table"} %}
{% set materialization_map = {"table": "table", "view": "view", "incremental": "table", "snapshot": "table"} %}

{% if node_resource_type|lower in ["model"] %}
{% if node_resource_type|lower in ["model", "snapshot"] %}

{# Append custom materializations to the list of standard materializations #}
{% do materialization_map.update(fromjson(var('custom_materializations_map', '{}'))) %}
2 changes: 1 addition & 1 deletion macros/snow-mask/apply-policy/unapply_masking_policy.sql
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@

{% if resource_type == "sources" %}
{{ dbt_snow_mask.apply_masking_policy_list_for_sources(meta_key,operation_type) }}
{% elif resource_type == "models" %}
{% elif resource_type|lower in ["models", "snapshots"] %}
{{ dbt_snow_mask.apply_masking_policy_list_for_models(meta_key,operation_type) }}
{% endif %}