diff --git a/docs/.gitbook/assets/feast-architecture-diagrams (1) (1) (1) (2) (3) (4) (3) (1) (1) (1) (1) (7).svg b/docs/.gitbook/assets/feast-architecture-diagrams (1) (1) (1) (2) (3) (4) (3) (1) (1) (1) (1) (7).svg new file mode 100644 index 0000000000..7335c131c4 --- /dev/null +++ b/docs/.gitbook/assets/feast-architecture-diagrams (1) (1) (1) (2) (3) (4) (3) (1) (1) (1) (1) (7).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (1).png b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (1).png similarity index 100% rename from docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (1).png rename to docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (1).png diff --git a/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2).png b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (2).png similarity index 100% rename from docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2).png rename to docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (2).png diff --git a/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (3).png b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (3).png new file mode 100644 index 0000000000..28efa1206a Binary files /dev/null and b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (3).png differ diff --git a/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (4).png b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (4).png new file mode 100644 index 0000000000..28efa1206a Binary files /dev/null and b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (4).png differ diff --git a/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (5).png b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (5).png new file mode 100644 index 0000000000..28efa1206a Binary files /dev/null and b/docs/.gitbook/assets/feast-landing-page-blog-post-page-5 (1) (1) (1) (2) (2) (5).png differ diff --git a/docs/.gitbook/assets/feast-on-aws-3- (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (3).png b/docs/.gitbook/assets/feast-on-aws-3- (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (3).png new file mode 100644 index 0000000000..e6de77dde9 Binary files /dev/null and b/docs/.gitbook/assets/feast-on-aws-3- (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (3).png differ diff --git a/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (1).png b/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (1).png similarity index 100% rename from docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (1).png rename to docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (1).png diff --git a/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2).png b/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2).png similarity index 100% rename from docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2).png rename to docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2).png diff --git a/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (3).png b/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (3).png new file mode 100644 index 0000000000..cd77f27cc4 Binary files /dev/null and b/docs/.gitbook/assets/image (4) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (2) (3).png differ diff --git a/docs/.gitbook/assets/image (6) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (1) (4).png b/docs/.gitbook/assets/image (6) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (1) (4).png new file mode 100644 index 0000000000..49670e2005 Binary files /dev/null and b/docs/.gitbook/assets/image (6) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (1) (4).png differ diff --git a/docs/.gitbook/assets/image (6) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (1) (5).png b/docs/.gitbook/assets/image (6) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (1) (5).png new file mode 100644 index 0000000000..49670e2005 Binary files /dev/null and b/docs/.gitbook/assets/image (6) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (1) (5).png differ diff --git a/docs/.gitbook/assets/point_in_time_join (1) (2) (2) (3) (3) (3) (3) (4).png b/docs/.gitbook/assets/point_in_time_join (1) (2) (2) (3) (3) (3) (3) (4).png new file mode 100644 index 0000000000..331a090d71 Binary files /dev/null and b/docs/.gitbook/assets/point_in_time_join (1) (2) (2) (3) (3) (3) (3) (4).png differ diff --git a/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (1).jpg b/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (1).jpg similarity index 100% rename from docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (1).jpg rename to docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (1).jpg diff --git a/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2).jpg b/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2).jpg similarity index 100% rename from docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2).jpg rename to docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (2).jpg diff --git a/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3).jpg b/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3).jpg similarity index 100% rename from docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3).jpg rename to docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3).jpg diff --git a/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (4).jpg b/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (4).jpg new file mode 100644 index 0000000000..b92ec6fed7 Binary files /dev/null and b/docs/.gitbook/assets/rsz_untitled23 (2) (2) (2) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (3) (4).jpg differ diff --git a/docs/.gitbook/assets/untitled-25-1- (2) (2) (2) (3) (3) (3) (3) (3) (3) (1) (4).jpg b/docs/.gitbook/assets/untitled-25-1- (2) (2) (2) (3) (3) (3) (3) (3) (3) (1) (4).jpg new file mode 100644 index 0000000000..93d010406b Binary files /dev/null and b/docs/.gitbook/assets/untitled-25-1- (2) (2) (2) (3) (3) (3) (3) (3) (3) (1) (4).jpg differ diff --git a/docs/README.md b/docs/README.md index 449639d63d..e52e33a39c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,7 +4,7 @@ Feast \(**Fea**ture **St**ore\) is an operational data system for managing and serving machine learning features to models in production. -![](.gitbook/assets/feast-architecture-diagrams%20%281%29%20%281%29%20%281%29%20%282%29%20%283%29%20%284%29%20%283%29%20%281%29%20%281%29%20%281%29%20%281%29%20%285%29.svg) +![](.gitbook/assets/feast-architecture-diagrams%20%281%29%20%281%29%20%281%29%20%282%29%20%283%29%20%284%29%20%283%29%20%281%29%20%281%29%20%281%29%20%281%29%20%286%29.svg) ## Problems Feast Solves diff --git a/docs/concepts/apply.md b/docs/concepts/apply.md index f061b543e5..2f02b2cf4c 100644 --- a/docs/concepts/apply.md +++ b/docs/concepts/apply.md @@ -2,27 +2,25 @@ The `apply` command is used to persist your feature definitions in the feature registry and to configure or provision the necessary infrastructure for your feature store to operate -### What happens during `feast apply` +### What happens during `feast apply`? #### 1. Scan the Feature Repository -Feast will scan Python files in your Feature Repository, and find all Feast object definitions, such as Feature Views, Entities, and Data Sources. +Feast will scan Python files in your feature repository, and find all Feast object definitions, such as feature views, entities, and data sources. #### 2. Update metadata -If all definitions look valid, Feast will sync the metadata about Feast objects to the Metadata Store. Metadata store is a tiny database storing most of the same information you have in the Feature Repository, plus some state in a more structured form. It is necessary mostly because the production feature serving infrastructure won't be able to access Python files in the Feature Repository at run time, but it will be able to efficiently and securely read the feature definitions from the Metadata Store. +If all definitions look valid, Feast will sync the metadata about Feast objects to the registry. The registry is a tiny database storing most of the same information you have in the feature repository. This step is necessary because the production feature serving infrastructure won't be able to access Python files in the feature repository at run time, but it will be able to efficiently and securely read the feature definitions from the registry. -#### 3. Create cloud infrastructure +### 3. Create cloud infrastructure -At this step, Feast CLI will create all necessary infrastructure for feature serving and materialization to work. What exactly gets created depends on what provider is configured to be used in `feature_store.yaml` in the Feature Repository. +Feast CLI will create all necessary infrastructure for feature serving and materialization to work. What exactly gets created depends on which provider is configured to be used in `feature_store.yaml` in the feature repository. -For example, for Local provider, it is as easy as creating a file on your local filesystem as a key-value store to serve feature data from. Local provider is most usable for local testing, no real production serving happens there. +For example, for the `local` provider, it is as easy as creating a sqlite database on disk as a key-value store to serve feature data from. The `local` provider is most usable for local testing, not production use. -A more interesting configuration is when we're configured Feast to use GCP provider and Cloud Datastore to store feature data. When you run `feast apply`, Feast will make sure you have valid credentials and create some metadata objects in the Datastore for each Feature View. - -Similarly, when using AWS, Feast will make sure that resources like DynamoDB tables are created for every Feature View. +A full featured configuration is to use the `gcp` provider and Cloud Datastore to store feature data. When you run `feast apply`, Feast will initialize Datastore based on your Feast objects \(like feature views\) in order to materialize and serve features. {% hint style="warning" %} -Since `feast deploy` \(when configured to use non-Local provider\) will create cloud infrastructure in your AWS or GCP account, it may incur some costs on your cloud bill. While we aim to design it in a way that Feast cloud resources don't cost much when not serving features, preferring "serverless" cloud services that bill per request, please refer to the specific Provider documentation to make sure there are no surprises. +Since `feast apply` \(when configured to use non-Local provider\) will create cloud infrastructure in your AWS or GCP account, it may incur some costs on your cloud bill. {% endhint %} diff --git a/docs/feast-on-kubernetes/advanced-1/security.md b/docs/feast-on-kubernetes/advanced-1/security.md index ede9573ec1..4d9b723be3 100644 --- a/docs/feast-on-kubernetes/advanced-1/security.md +++ b/docs/feast-on-kubernetes/advanced-1/security.md @@ -10,7 +10,7 @@ This page applies to Feast 0.7. The content may be out of date for Feast 0.8+ ### Overview -![Overview of Feast's Security Methods.](../../.gitbook/assets/untitled-25-1-%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%281%29%20%281%29.jpg) +![Overview of Feast's Security Methods.](../../.gitbook/assets/untitled-25-1-%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%281%29%20%282%29.jpg) Feast supports the following security methods: @@ -449,7 +449,7 @@ Authorization provides access control to FeatureTables and/or Features based on #### **Authorization API/Server** -![Feast Authorization Flow](../../.gitbook/assets/rsz_untitled23%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29.jpg) +![Feast Authorization Flow](../../.gitbook/assets/rsz_untitled23%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29.jpg) Feast delegates Authorization grants to an external Authorization Server that implements the [Authorization Open API specification](https://github.com/feast-dev/feast/blob/master/common/src/main/resources/api.yaml). diff --git a/docs/feast-on-kubernetes/concepts/architecture.md b/docs/feast-on-kubernetes/concepts/architecture.md index bac84d0027..f4cf23eb95 100644 --- a/docs/feast-on-kubernetes/concepts/architecture.md +++ b/docs/feast-on-kubernetes/concepts/architecture.md @@ -1,8 +1,6 @@ # Architecture - - -![](../../.gitbook/assets/image%20%286%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%282%29%20%281%29%20%283%29.png) +![](../../.gitbook/assets/image%20%286%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%282%29%20%281%29%20%282%29.png) ## Sequence description diff --git a/docs/feast-on-kubernetes/concepts/overview.md b/docs/feast-on-kubernetes/concepts/overview.md index ce59e10832..f82f4c4223 100644 --- a/docs/feast-on-kubernetes/concepts/overview.md +++ b/docs/feast-on-kubernetes/concepts/overview.md @@ -10,7 +10,7 @@ ### Concept Hierarchy -![](../../.gitbook/assets/image%20%284%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%281%29.png) +![](../../.gitbook/assets/image%20%284%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%281%29.png) Feast contains the following core concepts: diff --git a/docs/feast-on-kubernetes/concepts/stores.md b/docs/feast-on-kubernetes/concepts/stores.md index 8a910719c8..2e3bac5528 100644 --- a/docs/feast-on-kubernetes/concepts/stores.md +++ b/docs/feast-on-kubernetes/concepts/stores.md @@ -1,12 +1,12 @@ # Stores -In Feast, a store is a database that is populated with feature data that will ultimately be served to models. +In Feast, a store is a database that is populated with feature data that will ultimately be served to models. -### Offline \(Historical\) Store +## Offline \(Historical\) Store The offline store maintains historical copies of feature values. These features are grouped and stored in feature tables. During retrieval of historical data, features are queries from these feature tables in order to produce training datasets. -### Online Store +## Online Store The online store maintains only the latest values for a specific feature. @@ -18,5 +18,3 @@ The online store maintains only the latest values for a specific feature. Feast only supports a single online store in production {% endhint %} - - diff --git a/docs/feast-on-kubernetes/user-guide/define-and-ingest-features.md b/docs/feast-on-kubernetes/user-guide/define-and-ingest-features.md index b5697da3c6..5a7e7288ec 100644 --- a/docs/feast-on-kubernetes/user-guide/define-and-ingest-features.md +++ b/docs/feast-on-kubernetes/user-guide/define-and-ingest-features.md @@ -2,12 +2,11 @@ In order to retrieve features for both training and serving, Feast requires data being ingested into its offline and online stores. - Users are expected to already have either a batch or stream source with data stored in it, ready to be ingested into Feast. Once a feature table \(with the corresponding sources\) has been registered with Feast, it is possible to load data from this source into stores. The following depicts an example ingestion flow from a data source to the online store. -### Batch Source to Online Store +## Batch Source to Online Store ```python from feast import Client @@ -26,7 +25,7 @@ client.start_offline_to_online_ingestion( ) ``` -### Stream Source to Online Store +## Stream Source to Online Store ```python from feast import Client @@ -39,13 +38,13 @@ driver_ft = client.get_feature_table("driver_trips") client.start_stream_to_online_ingestion(driver_ft) ``` -### Batch Source to Offline Store +## Batch Source to Offline Store {% hint style="danger" %} Not supported in Feast 0.8 {% endhint %} -### Stream Source to Offline Store +## Stream Source to Offline Store {% hint style="danger" %} Not supported in Feast 0.8 diff --git a/docs/feast-on-kubernetes/user-guide/getting-training-features.md b/docs/feast-on-kubernetes/user-guide/getting-training-features.md index 0a24181fad..f80b650aa6 100644 --- a/docs/feast-on-kubernetes/user-guide/getting-training-features.md +++ b/docs/feast-on-kubernetes/user-guide/getting-training-features.md @@ -56,7 +56,7 @@ In the example below there are two tables \(or dataframes\): The user would like to have the driver features joined onto the entity dataframe to produce a training dataset that contains both the target \(trip\_completed\) and features \(average\_daily\_rides, maximum\_daily\_rides, rating\). This dataset will then be used to train their model. -![](../../.gitbook/assets/point_in_time_join%20%281%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%281%29.png) +![](../../.gitbook/assets/point_in_time_join%20%281%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%282%29.png) Feast is able to intelligently join feature data with different timestamps to a single entity dataframe. It does this through a point-in-time join as follows: diff --git a/docs/quickstart.md b/docs/quickstart.md index f6c6a7d2e8..a282d0b1fc 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -165,7 +165,7 @@ entity_df.head() ``` {% endcode %} -![](.gitbook/assets/feast-landing-page-blog-post-page-5%20%281%29%20%281%29%20%281%29%20%282%29.png) +![](.gitbook/assets/feast-landing-page-blog-post-page-5%20%281%29%20%281%29%20%281%29%20%282%29%20%282%29%20%283%29.png) This DataFrame represents the entity keys and timestamps that we want feature values for. We can pass this Entity DataFrame into Feast, and Feast will fetch point-in-time correct features for each row: