Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Remove PySpark dependency from Snowflake Offline Store #3388

Merged
merged 4 commits into from
Dec 14, 2022
Merged

fix: Remove PySpark dependency from Snowflake Offline Store #3388

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8f0947c
fix: Wrap pyspark import in try except for snowflake offline store
beauchbum Dec 14, 2022
43d6a87
fix: Capture exception
beauchbum Dec 14, 2022
9ed5e54
fix: update jinja template
beauchbum Dec 14, 2022
5a63e48
lint
adchia Dec 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion infra/templates/README.md.jinja2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ pip install feast
### 2. Create a feature repository
```commandline
feast init my_feature_repo
cd my_feature_repo
cd my_feature_repo/feature_repo
```

### 3. Register your feature definitions and set up your feature store
Expand Down
8 changes: 7 additions & 1 deletion sdk/python/feast/infra/offline_stores/snowflake.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
import pyarrow
from pydantic import Field, StrictStr
from pydantic.typing import Literal
from pyspark.sql import DataFrame, SparkSession
from pytz import utc

from feast import OnDemandFeatureView
Expand Down Expand Up @@ -64,6 +63,13 @@

raise FeastExtrasDependencyImportError("snowflake", str(e))

try:
from pyspark.sql import DataFrame, SparkSession
except ImportError as e:
from feast.errors import FeastExtrasDependencyImportError

raise FeastExtrasDependencyImportError("spark", str(e))

warnings.filterwarnings("ignore", category=DeprecationWarning)


Expand Down