Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add authentication option for snowflake connector #3039

Merged
merged 1 commit into from
Aug 10, 2022
Merged

feat: Add authentication option for snowflake connector #3039

merged 1 commit into from
Aug 10, 2022

Conversation

lsaiken
Copy link
Contributor

@lsaiken lsaiken commented Aug 8, 2022

Signed-off-by: Lisa Aiken laiken@manifold.ai

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

felixwang9817
felixwang9817 approved these changes Aug 8, 2022
View reviewed changes
Copy link
Collaborator

@felixwang9817 felixwang9817 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: felixwang9817, lsaiken

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@felixwang9817
Copy link
Collaborator

hey @lsaiken thanks for the PR! everything lgtm except there's a lint error that you'll need to fix

you can find more about how we handle linting here

@lsaiken lsaiken changed the title Add authentication option for snowflake connector feat: Add authentication option for snowflake connector Aug 8, 2022
@codecov-commenter
Copy link

codecov-commenter commented Aug 8, 2022 •
edited
Loading

Codecov Report

Merging #3039 (9289ff0) into master (e78d1d7) will increase coverage by 9.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3039      +/-   ##
==========================================
+ Coverage   67.59%   76.68%   +9.09%     
==========================================
  Files         168      198      +30     
  Lines       14712    16348    +1636     
==========================================
+ Hits         9944    12536    +2592     
+ Misses       4768     3812     -956
Flag Coverage Δ
integrationtests 67.44% <100.00%> (-0.15%) ⬇️
unittests 58.63% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...hon/feast/infra/offline_stores/snowflake_source.py 93.16% <ø> (+0.85%) ⬆️
sdk/python/feast/infra/utils/snowflake_utils.py 77.77% <ø> (ø)
sdk/python/feast/infra/offline_stores/snowflake.py 83.05% <100.00%> (+0.14%) ⬆️
sdk/python/feast/infra/online_stores/snowflake.py 98.16% <100.00%> (+0.03%) ⬆️
...offline_stores/contrib/spark_repo_configuration.py 100.00% <0.00%> (ø)
...s/contrib/spark_offline_store/tests/data_source.py 40.42% <0.00%> (ø)
.../online_stores/contrib/hbase_online_store/hbase.py 37.25% <0.00%> (ø)
...ib/trino_offline_store/test_config/manual_tests.py 33.33% <0.00%> (ø)
...b/cassandra_online_store/cassandra_online_store.py 2.63% <0.00%> (ø)
.../contrib/postgres_offline_store/postgres_source.py 49.01% <0.00%> (ø)
... and 98 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@sfc-gh-madkins
Copy link
Collaborator

@lsaiken did you run into any issues of having to authenticate an excessive amount?

See https://docs.snowflake.com/en/user-guide/admin-security-fed-auth-use.html#using-connection-caching-to-minimize-the-number-of-prompts-for-authentication-optional

sfc-gh-madkins
sfc-gh-madkins reviewed Aug 9, 2022
View reviewed changes
sdk/python/feast/infra/offline_stores/snowflake.py Outdated
@@ -91,6 +91,9 @@ class SnowflakeOfflineStoreConfig(FeastConfigBaseModel):
schema_: Optional[str] = Field(None, alias="schema")
""" Snowflake schema name """

authenticator_: Optional[str] = Field("", alias="authenticator")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was "authenticator" a reserved word?

Copy link
Contributor Author

@lsaiken lsaiken Aug 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's not a reserved keyword. I'll make the adjustment. Thanks for catching that!

To answer your question about excessive authentication, I didn't run into any issues having to authenticate an excessive amount. During testing, it only authenticates automatically through Okta when the snowflake.connector.connect commands are called.

@feast-ci-bot feast-ci-bot removed the lgtm label Aug 9, 2022
sfc-gh-madkins
sfc-gh-madkins reviewed Aug 9, 2022
View reviewed changes
sdk/python/feast/infra/offline_stores/snowflake.py Outdated
@@ -91,6 +91,9 @@ class SnowflakeOfflineStoreConfig(FeastConfigBaseModel):
schema_: Optional[str] = Field(None, alias="schema")
""" Snowflake schema name """

authenticator: Optional[str] = ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it on excessive authentication.

See following error in integration test:

"E snowflake.connector.errors.DatabaseError: 390139 (08001): None: Failed to connect to DB: ***.snowflakecomputing.com:443, The specified authenticator is not accepted by your Snowflake account configuration. Please contact your local system administrator to get the correct URL to use."

My guess is your code is always passing the authenticator kwarg, when it should only be passed if specified

authenticator: Optional[str] = ""
vs
authenticator: Optional[str] = None

would be my first guess

@lsaiken
feat: Add authentication option for snowflake connector
9289ff0 
Signed-off-by: Lisa Aiken <laiken@manifold.ai>
@felixwang9817
Copy link
Collaborator

/lgtm

@felixwang9817
Copy link
Collaborator

thanks @sfc-gh-madkins for the help here!

@feast-ci-bot feast-ci-bot merged commit 74c75f1 into feast-dev:master Aug 10, 2022
@lsaiken
Copy link
Contributor Author

lsaiken commented Aug 10, 2022

thanks @sfc-gh-madkins and @felixwang9817!

franciscojavierarceo pushed a commit to franciscojavierarceo/feast that referenced this pull request Aug 13, 2022
@lsaiken @franciscojavierarceo
feat: Add authentication option for snowflake connector (feast-dev#3039)
5aa3d00 
Signed-off-by: Lisa Aiken <laiken@manifold.ai>

Signed-off-by: Lisa Aiken <laiken@manifold.ai>
Signed-off-by: Francisco Javier Arceo <arceofrancisco@gmail.com>
kevjumba pushed a commit that referenced this pull request Aug 25, 2022
@feast-ci-bot
chore(release): release 0.24.0
c1c71da 
# [0.24.0](v0.23.0...v0.24.0) (2022-08-25)

### Bug Fixes

* Check if on_demand_feature_views is an empty list rather than None for snowflake provider ([#3046](#3046)) ([9b05e65](9b05e65))
* FeatureStore.apply applies BatchFeatureView correctly ([#3098](#3098)) ([41be511](41be511))
* Fix Feast Java inconsistency with int64 serialization vs python ([#3031](#3031)) ([4bba787](4bba787))
* Fix feature service inference logic ([#3089](#3089)) ([4310ed7](4310ed7))
* Fix field mapping logic during feature inference ([#3067](#3067)) ([cdfa761](cdfa761))
* Fix incorrect on demand feature view diffing and improve Java tests ([#3074](#3074)) ([0702310](0702310))
* Fix Java helm charts to work with refactored logic. Fix FTS image ([#3105](#3105)) ([2b493e0](2b493e0))
* Fix on demand feature view output in feast plan + Web UI crash ([#3057](#3057)) ([bfae6ac](bfae6ac))
* Fix release workflow to release 0.24.0 ([#3138](#3138)) ([a69aaae](a69aaae))
* Fix Spark offline store type conversion to arrow ([#3071](#3071)) ([b26566d](b26566d))
* Fixing Web UI, which fails for the SQL registry ([#3028](#3028)) ([64603b6](64603b6))
* Force Snowflake Session to Timezone UTC ([#3083](#3083)) ([9f221e6](9f221e6))
* Make infer dummy entity join key idempotent ([#3115](#3115)) ([1f5b1e0](1f5b1e0))
* More explicit error messages ([#2708](#2708)) ([e4d7afd](e4d7afd))
* Parse inline data sources ([#3036](#3036)) ([c7ba370](c7ba370))
* Prevent overwriting existing file during `persist` ([#3088](#3088)) ([69af21f](69af21f))
* Register BatchFeatureView in feature repos correctly ([#3092](#3092)) ([b8e39ea](b8e39ea))
* Return an empty infra object from sql registry when it doesn't exist ([#3022](#3022)) ([8ba87d1](8ba87d1))
* Teardown tables for Snowflake Materialization testing ([#3106](#3106)) ([0a0c974](0a0c974))
* UI error when saved dataset is present in registry. ([#3124](#3124)) ([83cf753](83cf753))
* Update sql.py ([#3096](#3096)) ([2646a86](2646a86))
* Updated snowflake template ([#3130](#3130)) ([f0594e1](f0594e1))

### Features

* Add authentication option for snowflake connector ([#3039](#3039)) ([74c75f1](74c75f1))
* Add Cassandra/AstraDB online store contribution ([#2873](#2873)) ([feb6cb8](feb6cb8))
* Add Snowflake materialization engine ([#2948](#2948)) ([f3b522b](f3b522b))
* Adding saved dataset capabilities for Postgres  ([#3070](#3070)) ([d3253c3](d3253c3))
* Allow passing repo config path via flag ([#3077](#3077)) ([0d2d951](0d2d951))
* Contrib azure provider with synapse/mssql offline store and Azure registry store ([#3072](#3072)) ([9f7e557](9f7e557))
* Custom Docker image for Bytewax batch materialization ([#3099](#3099)) ([cdd1b07](cdd1b07))
* Feast AWS Athena offline store (again) ([#3044](#3044)) ([989ce08](989ce08))
* Implement spark offline store `offline_write_batch` method ([#3076](#3076)) ([5b0cc87](5b0cc87))
* Initial Bytewax materialization engine ([#2974](#2974)) ([55c61f9](55c61f9))
* Refactor feature server helm charts to allow passing feature_store.yaml in environment variables ([#3113](#3113)) ([85ee789](85ee789))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sfc-gh-madkins sfc-gh-madkins sfc-gh-madkins left review comments

@felixwang9817 felixwang9817 felixwang9817 approved these changes

Assignees

@felixwang9817 felixwang9817

Labels
approved lgtm ok-to-test size/S
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lsaiken @feast-ci-bot @felixwang9817 @codecov-commenter @sfc-gh-madkins