Could not import module 'feast.infra.offline_stores.snowflake' #3580

wjhhuizi · 2023-03-31T18:32:44Z

Expected Behavior

Feast should correctly import 'feast.infra.online_stores.snowflake' or 'feast.infra.offline_stores.snowflake' modules when being invoked in CLI or SDK.

Current Behavior

When running any command with FEAST CLI, if in feature_store.yaml we provide

offline_store:
    type: snowflake.offline

as our offline storage, the console would return following error stack trace and exit:

Feast is an open source project that collects anonymized error reporting and usage statistics. To opt out or learn more see https://docs.feast.dev/reference/usage
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/feast/importer.py", line 26, in import_class
    module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/feast/infra/online_stores/snowflake.py", line 15, in <module>
    from feast.infra.utils.snowflake.snowflake_utils import (
  File "/usr/local/lib/python3.8/site-packages/feast/infra/utils/snowflake/snowflake_utils.py", line 28, in <module>
    import snowflake.connector
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/__init__.py", line 16, in <module>
    from .connection import SnowflakeConnection
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/connection.py", line 25, in <module>
    from . import errors, proxy
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/errors.py", line 18, in <module>
    from .telemetry_oob import TelemetryService
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/telemetry_oob.py", line 20, in <module>
    from .vendored import requests
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/vendored/requests/__init__.py", line 119, in <module>
    from ..urllib3.contrib import pyopenssl
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/contrib/pyopenssl.py", line 50, in <module>
    import OpenSSL.SSL
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/crypto.py", line 3268, in <module>
    _lib.OpenSSL_add_all_algorithms()
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/feast", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/feast/cli.py", line 178, in ui
    store = FeatureStore(repo_path=str(repo), fs_yaml_file=fs_yaml_file)
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 362, in wrapper
    raise exc.with_traceback(traceback)
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 348, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/feast/feature_store.py", line 152, in __init__
    self.config = load_repo_config(self.repo_path, fs_yaml_file)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 548, in load_repo_config
    c = RepoConfig(**raw_config)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 184, in __init__
    super().__init__(**data)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1050, in pydantic.main.validate_model
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 362, in wrapper
    raise exc.with_traceback(traceback)
  File "/usr/local/lib/python3.8/site-packages/feast/usage.py", line 348, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 346, in _validate_online_store_config
    online_config_class = get_online_config_from_type(online_store_type)
  File "/usr/local/lib/python3.8/site-packages/feast/repo_config.py", line 518, in get_online_config_from_type
    return import_class(module_name, config_class_name, config_class_name)
  File "/usr/local/lib/python3.8/site-packages/feast/importer.py", line 31, in import_class
    raise FeastModuleImportError(module_name, class_name) from e
feast.errors.FeastModuleImportError: Could not import module 'feast.infra.online_stores.snowflake' while attempting to load class 'SnowflakeOnlineStoreConfig'

This issue is currently existing in version 0.27.0 all the way up to 0.30.2.

There is a relevant issue (#2560) which marked as closed but the fix was not effective.

Steps to reproduce

Provide the feature_store.yaml and run any feast cli command with version of 0.27.0 or above

project: ProjectName

registry:
    registry_type: sql
    path: mysql://XXXX:@hostname:3306/mysql
    cache_ttl_seconds: 600

provider: local

online_store:
    type: sqlite
    path: data/online_store.db

offline_store:
    type: snowflake.offline
    account: xxxxxx.region.privatelink
    user: XXX_XXX_XXX
    password: 'P@SSWORD123$'
    role: 'XXX_XXX_XXXXXX_XXX'
    warehouse: XX_XX_XXXX_XXXX
    database: 'XXXX_XX_XXXXX_XX'
    schema: 'XXXX_XXX_XXXXXX_XX'
    
entity_key_serialization_version: 2

IF no online_store provided, the error will then reflecting as Could not import module 'feast.infra.online_stores.snowflake'

Specifications

Version: 0.27.0, 0.28.0, 0.29.0, 0.30.2
Platform: Linux
Subsystem: Linux

Possible Solution

The text was updated successfully, but these errors were encountered:

wjhhuizi · 2023-03-31T18:43:43Z

AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

This could be the direct cause of the issue.

Kavee30 · 2023-03-31T18:45:11Z

Hi @achals
Need your input in fixing this bug.

wjhhuizi · 2023-03-31T21:04:36Z

After did some research and did pip3 install cryptography==38.0.4, another error raised

feast.errors.FeastClassImportError: Could not import class 'SqlRegistryConfig' from module 'feast.infra.registry.sql'

sfc-gh-madkins · 2023-04-05T14:41:49Z

@wjhhuizi does it work with a different registry than SQL?

sfc-gh-madkins · 2023-04-05T19:31:10Z

I think #3587 should solve this

davidschuler-8451 · 2023-04-05T19:33:11Z

After did some research and did pip3 install cryptography==38.0.4, another error raised

feast.errors.FeastClassImportError: Could not import class 'SqlRegistryConfig' from module 'feast.infra.registry.sql'

this is the same issue as #3544. I just opened a PR to address that but need some feedback on if it's the right approach or not #3586

Kavee30 · 2023-04-06T09:59:01Z

@davidschuler-8451 @sfc-gh-madkins With feast version - 0.30.2, even the command 'feast init -t snowflake' throws the same error

wjhhuizi · 2023-04-12T19:27:25Z

It seems the PR #3587 was closed, what will happen next? @sfc-gh-madkins

sfc-gh-madkins · 2023-04-12T20:32:11Z

I cant confirm that PR would have solved the issued. I need more details than the error you are providing.

…

On Wed, Apr 12, 2023 at 2:27 PM Michael Wu ***@***.***> wrote: It seems the PR #3587 <#3587> was closed, what will happen next? @sfc-gh-madkins <https://github.com/sfc-gh-madkins> — Reply to this email directly, view it on GitHub <#3580 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATSRCU4VIYQVSBODFYWANDDXA362TANCNFSM6AAAAAAWO7X4MY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

wjhhuizi · 2023-04-13T17:05:37Z

Please let me know what type of detail do you need, let me see if I can help to provide :-)

sfc-gh-madkins · 2023-04-24T16:38:05Z

#3608

sfc-gh-madkins · 2023-06-13T01:08:16Z

Matus -- Can you provide a python version & reqs.txt you are trying this on so I can recreate the error?

…

On Mon, Jun 12, 2023 at 3:41 AM Matúš Chládek ***@***.***> wrote: Did 0.31.1 <v0.31.0...v0.31.1> resolve this for anyone? I am still getting same on 0.31.1 — Reply to this email directly, view it on GitHub <#3580 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATSRCU3NPELZSG3JH5WKA43XK3I2XANCNFSM6AAAAAAWO7X4MY> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

wjhhuizi added kind/bug priority/p2 labels Mar 31, 2023

wjhhuizi changed the title ~~Could not import module 'feast.infra.online_stores.snowflake'~~ Could not import module 'feast.infra.offline_stores.snowflake' Mar 31, 2023

sfc-gh-madkins mentioned this issue Apr 5, 2023

chore: Upgrade Snowflake to 2.9.0 #3587

Closed

sfc-gh-madkins mentioned this issue Apr 23, 2023

chore: Upgrade snowflake version to 3.0.0 #3608

Merged

sfc-gh-madkins closed this as completed Apr 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Could not import module 'feast.infra.offline_stores.snowflake' #3580

Could not import module 'feast.infra.offline_stores.snowflake' #3580

wjhhuizi commented Mar 31, 2023

wjhhuizi commented Mar 31, 2023

Kavee30 commented Mar 31, 2023

wjhhuizi commented Mar 31, 2023

sfc-gh-madkins commented Apr 5, 2023

sfc-gh-madkins commented Apr 5, 2023

davidschuler-8451 commented Apr 5, 2023

Kavee30 commented Apr 6, 2023

wjhhuizi commented Apr 12, 2023

sfc-gh-madkins commented Apr 12, 2023 via email

wjhhuizi commented Apr 13, 2023

sfc-gh-madkins commented Apr 24, 2023

sfc-gh-madkins commented Jun 13, 2023 via email

Could not import module 'feast.infra.offline_stores.snowflake' #3580

Could not import module 'feast.infra.offline_stores.snowflake' #3580

Comments

wjhhuizi commented Mar 31, 2023

Expected Behavior

Current Behavior

Steps to reproduce

Specifications

Possible Solution

wjhhuizi commented Mar 31, 2023

Kavee30 commented Mar 31, 2023

wjhhuizi commented Mar 31, 2023

sfc-gh-madkins commented Apr 5, 2023

sfc-gh-madkins commented Apr 5, 2023

davidschuler-8451 commented Apr 5, 2023

Kavee30 commented Apr 6, 2023

wjhhuizi commented Apr 12, 2023

sfc-gh-madkins commented Apr 12, 2023 via email

wjhhuizi commented Apr 13, 2023

sfc-gh-madkins commented Apr 24, 2023

sfc-gh-madkins commented Jun 13, 2023 via email