feat: masking policy in v2 sdk (#1777)

* masking policy in v2 sdk * add more tests and docs * add back in region check * linting * linting * linting * linting * fix unit test * update docs * address comments * address comments * remove tag ddl type * Update docs/index.md Co-authored-by: Nathan Gaberel <nathan.gaberel@snowflake.com> * Update templates/index.md.tmpl Co-authored-by: Nathan Gaberel <nathan.gaberel@snowflake.com> * fix PR comments * chore: misc linting errors (#1779) * fix misc linting errors * linting fixes * int tests * cleanup env vars * fix int test for masking policy tag * fix linting errors * remove useless test * skip email notification test * fix int test * fix int test * fix int test * adjust timeout * update docs * disable-if-return * add in temp warehouse * if-return revert * update-docs --------- Co-authored-by: Nathan Gaberel <nathan.gaberel@snowflake.com>
Snowflake-Labs · May 9, 2023 · 6978c42 · 6978c42
1 parent 6feeb9c
commit 6978c42
Show file tree

Hide file tree

Showing 118 changed files with 3,336 additions and 1,215 deletions.
diff --git a/.golangci.yml b/.golangci.yml
@@ -11,7 +11,10 @@ linters-settings:
     packages:
       - github.com/pkg/error
       - io/ioutil
-
+  revive:
+    rules:
+      - name: if-return
+        disabled: true
 linters:
   disable-all: true
   enable:

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -30,6 +30,18 @@ export SNOWFLAKE_REGION=us-west-2
 export TF_ACC=true
 ```
 
+You can also read the config from a `~/.snowflake/config` file, although you will still need to set `TF_ACC` to true.
+
+
+~/.snowflake/config
+```sh
+[default]
+account='TESTACCOUNT'
+user='TEST_USER'
+password='hunter2'
+role='ACCOUNTADMIN'
+```
+
 **Note: PRs for new resources will not be accepted without passing acceptance tests.**
 
 For the Terraform resources, there are 3 levels of testing - internal, unit and acceptance tests.
@@ -86,4 +98,4 @@ Releases will be performed as needed, typically once every 1-2 weeks. If your ch
 
 Releases are done by [goreleaser](https://goreleaser.com/) and run by our make files. There two goreleaser configs, `.goreleaser.yml` for regular releases and `.goreleaser.prerelease.yml` for doing prereleases (for testing).
 
-Releases are [published to the terraform registry](https://registry.terraform.io/providers/chanzuckerberg/snowflake/latest), which requires that releases by signed.
+Releases are [published to the terraform registry](https://registry.terraform.io/providers/chanzuckerberg/snowflake/latest), which requires that releases by signed.
diff --git a/Makefile b/Makefile
@@ -62,7 +62,7 @@ test:  ## run the tests (except sdk tests)
 .PHONY: test
 
 test-acceptance: ## runs all tests, including the acceptance tests which create and destroys real resources
-	SKIP_MANAGED_ACCOUNT_TEST=1 TF_ACC=1 $(go_test) -v -coverprofile=coverage.txt -covermode=atomic $(TESTARGS) ./...
+	SKIP_MANAGED_ACCOUNT_TEST=1 SKIP_EMAIL_INTEGRATION_TESTS=1 TF_ACC=1 $(go_test) -timeout 900s -v -coverprofile=coverage.txt -covermode=atomic $(TESTARGS) ./...
 .PHONY: test-acceptance
 
 deps:

diff --git a/docs/index.md b/docs/index.md
@@ -13,11 +13,8 @@ Coverage is focused on part of Snowflake related to access control.
 
 ```terraform
 provider "snowflake" {
-  // required
-  username = "..."
-  account  = "..." # the Snowflake account identifier
-
-  // optional, exactly one must be set
+  account  = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT env var
+  username = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var
   password               = "..."
   oauth_access_token     = "..."
   private_key_path       = "..."
@@ -35,6 +32,11 @@ provider "snowflake" {
   host      = "..."
   warehouse = "..."
 }
+
+
+provider snowflake {
+  profile = "securityadmin"
+}
 ```
 
 ## Configuration Schema
@@ -44,13 +46,9 @@ provider "snowflake" {
 <!-- schema generated by tfplugindocs -->
 ## Schema
 
-### Required
-
-- `account` (String) The name of the Snowflake account. Can also come from the `SNOWFLAKE_ACCOUNT` environment variable.
-- `username` (String) Username for username+password authentication. Can come from the `SNOWFLAKE_USER` environment variable.
-
 ### Optional
 
+- `account` (String) The name of the Snowflake account. Can also come from the `SNOWFLAKE_ACCOUNT` environment variable. Required unless using profile.
 - `browser_auth` (Boolean) Required when `oauth_refresh_token` is used. Can be sourced from `SNOWFLAKE_USE_BROWSER_AUTH` environment variable.
 - `host` (String) Supports passing in a custom host value to the snowflake go driver for use with privatelink.
 - `insecure_mode` (Boolean) If true, bypass the Online Certificate Status Protocol (OCSP) certificate revocation check. IMPORTANT: Change the default value for testing or emergency situations only.
@@ -65,9 +63,11 @@ provider "snowflake" {
 - `private_key` (String, Sensitive) Private Key for username+private-key auth. Cannot be used with `browser_auth` or `password`. Can be sourced from `SNOWFLAKE_PRIVATE_KEY` environment variable.
 - `private_key_passphrase` (String, Sensitive) Supports the encryption ciphers aes-128-cbc, aes-128-gcm, aes-192-cbc, aes-192-gcm, aes-256-cbc, aes-256-gcm, and des-ede3-cbc
 - `private_key_path` (String, Sensitive) Path to a private key for using keypair authentication. Cannot be used with `browser_auth`, `oauth_access_token` or `password`. Can be sourced from `SNOWFLAKE_PRIVATE_KEY_PATH` environment variable.
+- `profile` (String) Sets the profile to read from ~/.snowflake/config file.
 - `protocol` (String) Support custom protocols to snowflake go driver. Can be sourced from `SNOWFLAKE_PROTOCOL` environment variable.
 - `region` (String) [Snowflake region](https://docs.snowflake.com/en/user-guide/intro-regions.html) to use.  Required if using the [legacy format for the `account` identifier](https://docs.snowflake.com/en/user-guide/admin-account-identifier.html#format-2-legacy-account-locator-in-a-region) in the form of `<cloud_region_id>.<cloud>`. Can be sourced from the `SNOWFLAKE_REGION` environment variable.
 - `role` (String) Snowflake role to use for operations. If left unset, default role for user will be used. Can be sourced from the `SNOWFLAKE_ROLE` environment variable.
+- `username` (String) Username for username+password authentication. Can come from the `SNOWFLAKE_USER` environment variable. Required unless using profile.
 - `warehouse` (String) Sets the default warehouse. Optional. Can be sourced from SNOWFLAKE_WAREHOUSE environment variable.
 
 ## Authentication
@@ -79,8 +79,9 @@ The Snowflake provider support multiple ways to authenticate:
 * OAuth Refresh Token
 * Browser Auth
 * Private Key
+* Config File
 
-In all cases account and region are required.
+In all cases account and username are required.
 
 ### Keypair Authentication Environment Variables
 
@@ -156,3 +157,28 @@ If you choose to use Username and Password Authentication, export these credenti
 export SNOWFLAKE_USER='...'
 export SNOWFLAKE_PASSWORD='...'
 ```
+
+### Config File
+
+If you choose to use a config file, the optional `profile` attribute specifies the profile to use from the config file. If no profile is specified, the default profile is used. The Snowflake config file lives at `~/.snowflake/config` and uses [TOML](https://toml.io/) format. You can override this location by setting the `SNOWFLAKE_CONFIG_PATH` environment variable. If no username and account are specified, the provider will fall back to reading the config file.
+
+```shell
+[default]
+account='TESTACCOUNT'
+user='TEST_USER'
+password='hunter2'
+role='ACCOUNTADMIN'
+
+[securityadmin]
+account='TESTACCOUNT'
+user='TEST_USER'
+password='hunter2'
+role='SECURITYADMIN'
+```
+
+## Order Precedence
+
+The Snowflake provider will use the following order of precedence when determining which credentials to use:
+1) Provider Configuration
+2) Environment Variables
+3) Config File
diff --git a/docs/resources/email_notification_integration.md b/docs/resources/email_notification_integration.md
@@ -0,0 +1,48 @@
+---
+# generated by https://github.com/hashicorp/terraform-plugin-docs
+page_title: "snowflake_email_notification_integration Resource - terraform-provider-snowflake"
+subcategory: ""
+description: |-
+  
+---
+
+# snowflake_email_notification_integration (Resource)
+
+
+
+## Example Usage
+
+```terraform
+resource "snowflake_email_notification_integration" "email_int" {
+  name    = "notification"
+  comment = "A notification integration."
+
+  enabled   = true
+  allowed_recipients = ["john.doe@gmail.com"]
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Required
+
+- `allowed_recipients` (Set of String) List of email addresses that should receive notifications.
+- `enabled` (Boolean)
+- `name` (String)
+
+### Optional
+
+- `comment` (String) A comment for the email integration.
+
+### Read-Only
+
+- `id` (String) The ID of this resource.
+
+## Import
+
+Import is supported using the following syntax:
+
+```shell
+terraform import snowflake_email_notification_integration.example name
+```
diff --git a/docs/resources/masking_policy.md b/docs/resources/masking_policy.md
@@ -13,13 +13,28 @@ description: |-
 ## Example Usage
 
 ```terraform
-resource "snowflake_masking_policy" "example_masking_policy" {
-  name               = "EXAMPLE_MASKING_POLICY"
+resource "snowflake_masking_policy" "test" {
+ name               = "EXAMPLE_MASKING_POLICY"
   database           = "EXAMPLE_DB"
   schema             = "EXAMPLE_SCHEMA"
-  value_data_type    = "string"
-  masking_expression = "case when current_role() in ('ANALYST') then val else sha2(val, 512) end"
-  return_data_type   = "string"
+	signature {
+		column {
+			name = "val"
+			type = "VARCHAR"
+		}
+	}
+  masking_expression = <<-EOF
+    case 
+      when current_role() in ('ROLE_A') then 
+        val 
+      when is_role_in_session( 'ROLE_B' ) then 
+        'ABC123'
+      else
+        '******'
+    end
+  EOF
+
+  return_data_type = "VARCHAR"
 }
 ```
 
@@ -33,17 +48,35 @@ resource "snowflake_masking_policy" "example_masking_policy" {
 - `name` (String) Specifies the identifier for the masking policy; must be unique for the database and schema in which the masking policy is created.
 - `return_data_type` (String) Specifies the data type to return.
 - `schema` (String) The schema in which to create the masking policy.
-- `value_data_type` (String) Specifies the data type to mask.
+- `signature` (Block List, Min: 1, Max: 1) The signature for the masking policy; specifies the input columns and data types to evaluate at query runtime. (see [below for nested schema](#nestedblock--signature))
 
 ### Optional
 
 - `comment` (String) Specifies a comment for the masking policy.
+- `exempt_other_policies` (Boolean) Specifies whether the row access policy or conditional masking policy can reference a column that is already protected by a masking policy.
+- `if_not_exists` (Boolean) Prevent overwriting a previous masking policy with the same name.
+- `or_replace` (Boolean) Whether to override a previous masking policy with the same name.
 
 ### Read-Only
 
 - `id` (String) The ID of this resource.
 - `qualified_name` (String) Specifies the qualified identifier for the masking policy.
 
+<a id="nestedblock--signature"></a>
+### Nested Schema for `signature`
+
+Required:
+
+- `column` (Block List, Min: 1) (see [below for nested schema](#nestedblock--signature--column))
+
+<a id="nestedblock--signature--column"></a>
+### Nested Schema for `signature.column`
+
+Required:
+
+- `name` (String) Specifies the column name to mask.
+- `type` (String) Specifies the column type to mask.
+
 ## Import
 
 Import is supported using the following syntax:

diff --git a/docs/resources/tag_masking_policy_association.md b/docs/resources/tag_masking_policy_association.md
@@ -3,12 +3,12 @@
 page_title: "snowflake_tag_masking_policy_association Resource - terraform-provider-snowflake"
 subcategory: ""
 description: |-
-  
+  Attach a masking policy to a tag. Requires a current warehouse to be set. Either with SNOWFLAKE_WAREHOUSE env variable or in current session. If no warehouse is provided, a temporary warehouse will be created.
 ---
 
 # snowflake_tag_masking_policy_association (Resource)
 
-
+Attach a masking policy to a tag. Requires a current warehouse to be set. Either with SNOWFLAKE_WAREHOUSE env variable or in current session. If no warehouse is provided, a temporary warehouse will be created.
 
 ## Example Usage
 

diff --git a/examples/provider/provider.tf b/examples/provider/provider.tf
@@ -1,9 +1,6 @@
 provider "snowflake" {
-  // required
-  username = "..."
-  account  = "..." # the Snowflake account identifier
-
-  // optional, exactly one must be set
+  account  = "..." # required if not using profile. Can also be set via SNOWFLAKE_ACCOUNT env var
+  username = "..." # required if not using profile or token. Can also be set via SNOWFLAKE_USER env var
   password               = "..."
   oauth_access_token     = "..."
   private_key_path       = "..."
@@ -21,3 +18,8 @@ provider "snowflake" {
   host      = "..."
   warehouse = "..."
 }
+
+
+provider snowflake {
+  profile = "securityadmin"
+}
diff --git a/examples/resources/snowflake_email_notification_integration/resource.tf b/examples/resources/snowflake_email_notification_integration/resource.tf
@@ -3,6 +3,5 @@ resource "snowflake_email_notification_integration" "email_int" {
   comment = "A notification integration."
 
   enabled   = true
-  allowed_recipients = ['john.doe@gmail.com']
-
+  allowed_recipients = ["john.doe@gmail.com"]
 }
diff --git a/examples/resources/snowflake_masking_policy/resource.tf b/examples/resources/snowflake_masking_policy/resource.tf
@@ -1,8 +1,23 @@
-resource "snowflake_masking_policy" "example_masking_policy" {
-  name               = "EXAMPLE_MASKING_POLICY"
+resource "snowflake_masking_policy" "test" {
+ name               = "EXAMPLE_MASKING_POLICY"
   database           = "EXAMPLE_DB"
   schema             = "EXAMPLE_SCHEMA"
-  value_data_type    = "string"
-  masking_expression = "case when current_role() in ('ANALYST') then val else sha2(val, 512) end"
-  return_data_type   = "string"
+	signature {
+		column {
+			name = "val"
+			type = "VARCHAR"
+		}
+	}
+  masking_expression = <<-EOF
+    case 
+      when current_role() in ('ROLE_A') then 
+        val 
+      when is_role_in_session( 'ROLE_B' ) then 
+        'ABC123'
+      else
+        '******'
+    end
+  EOF
+
+  return_data_type = "VARCHAR"
 }
diff --git a/go.mod b/go.mod
@@ -5,6 +5,7 @@ go 1.19
 require (
 	github.com/DATA-DOG/go-sqlmock v1.5.0
 	github.com/Pallinder/go-randomdata v1.2.0
+	github.com/buger/jsonparser v1.1.1
 	github.com/hashicorp/terraform-plugin-docs v0.14.1
 	github.com/hashicorp/terraform-plugin-sdk/v2 v2.26.1
 	github.com/jmoiron/sqlx v1.3.5
@@ -48,6 +49,7 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/s3 v1.30.6 // indirect
 	github.com/aws/smithy-go v1.13.5 // indirect
 	github.com/bgentry/speakeasy v0.1.0 // indirect
+	github.com/brianvoe/gofakeit/v6 v6.21.0 // indirect
 	github.com/danieljoos/wincred v1.1.2 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/dvsekhvalnov/jose2go v1.5.0 // indirect
@@ -99,6 +101,7 @@ require (
 	github.com/mitchellh/reflectwalk v1.0.2 // indirect
 	github.com/mtibben/percent v0.2.1 // indirect
 	github.com/oklog/run v1.1.0 // indirect
+	github.com/pelletier/go-toml/v2 v2.0.7 // indirect
 	github.com/pierrec/lz4/v4 v4.1.17 // indirect
 	github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect

diff --git a/go.sum b/go.sum
@@ -88,6 +88,10 @@ github.com/aws/smithy-go v1.13.5 h1:hgz0X/DX0dGqTYpGALqXJoRKRj5oQ7150i5FdTePzO8=
 github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
 github.com/bgentry/speakeasy v0.1.0 h1:ByYyxL9InA1OWqxJqqp2A5pYHUrCiAL6K3J+LKSsQkY=
 github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
+github.com/brianvoe/gofakeit/v6 v6.21.0 h1:tNkm9yxEbpuPK8Bx39tT4sSc5i9SUGiciLdNix+VDQY=
+github.com/brianvoe/gofakeit/v6 v6.21.0/go.mod h1:Ow6qC71xtwm79anlwKRlWZW6zVq9D2XHE4QSSMP/rU8=
+github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs=
+github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
 github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
 github.com/danieljoos/wincred v1.1.2 h1:QLdCxFs1/Yl4zduvBdcHB8goaYk9RARS2SgLLRuAyr0=
 github.com/danieljoos/wincred v1.1.2/go.mod h1:GijpziifJoIBfYh+S7BbkdUTU4LfM+QnGqR5Vl2tAx0=
@@ -279,6 +283,8 @@ github.com/mtibben/percent v0.2.1/go.mod h1:KG9uO+SZkUp+VkRHsCdYQV3XSZrrSpR3O9ib
 github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
 github.com/oklog/run v1.1.0 h1:GEenZ1cK0+q0+wsJew9qUg/DyD8k3JzYsZAi5gYi2mA=
 github.com/oklog/run v1.1.0/go.mod h1:sVPdnTZT1zYwAJeCMu2Th4T21pA3FPOQRfWjQlk7DVU=
+github.com/pelletier/go-toml/v2 v2.0.7 h1:muncTPStnKRos5dpVKULv2FVd4bMOhNePj9CjgDb8Us=
+github.com/pelletier/go-toml/v2 v2.0.7/go.mod h1:eumQOmlWiOPt5WriQQqoM5y18pDHwha2N+QD+EUNTek=
 github.com/pierrec/lz4/v4 v4.1.17 h1:kV4Ip+/hUBC+8T6+2EgburRtkE9ef4nbY3f4dFhGjMc=
 github.com/pierrec/lz4/v4 v4.1.17/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
 github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 h1:KoWmjvw+nsYOo29YJK9vDA65RGE3NrOnUtO7a+RF9HU=
@@ -321,6 +327,7 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
 github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
 github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
 github.com/vmihailenco/msgpack v3.3.3+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=