Skip to content

Commit

Permalink
GitBook: [master] 58 pages and 25 assets modified
Browse files Browse the repository at this point in the history
  • Loading branch information
woop authored and gitbook-bot committed Apr 21, 2021
1 parent 61fd905 commit 1c5e2aa
Show file tree
Hide file tree
Showing 27 changed files with 23 additions and 29 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Feast \(**Fea**ture **St**ore\) is an operational data system for managing and serving machine learning features to models in production.

![](.gitbook/assets/feast-architecture-diagrams%20%281%29%20%281%29%20%281%29%20%282%29%20%283%29%20%284%29%20%283%29%20%281%29%20%281%29%20%281%29%20%281%29%20%285%29.svg)
![](.gitbook/assets/feast-architecture-diagrams%20%281%29%20%281%29%20%281%29%20%282%29%20%283%29%20%284%29%20%283%29%20%281%29%20%281%29%20%281%29%20%281%29%20%286%29.svg)

## Problems Feast Solves

Expand Down
18 changes: 8 additions & 10 deletions docs/concepts/apply.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,25 @@

The `apply` command is used to persist your feature definitions in the feature registry and to configure or provision the necessary infrastructure for your feature store to operate

### What happens during `feast apply`
### What happens during `feast apply`?

#### 1. Scan the Feature Repository

Feast will scan Python files in your Feature Repository, and find all Feast object definitions, such as Feature Views, Entities, and Data Sources.
Feast will scan Python files in your feature repository, and find all Feast object definitions, such as feature views, entities, and data sources.

#### 2. Update metadata

If all definitions look valid, Feast will sync the metadata about Feast objects to the Metadata Store. Metadata store is a tiny database storing most of the same information you have in the Feature Repository, plus some state in a more structured form. It is necessary mostly because the production feature serving infrastructure won't be able to access Python files in the Feature Repository at run time, but it will be able to efficiently and securely read the feature definitions from the Metadata Store.
If all definitions look valid, Feast will sync the metadata about Feast objects to the registry. The registry is a tiny database storing most of the same information you have in the feature repository. This step is necessary because the production feature serving infrastructure won't be able to access Python files in the feature repository at run time, but it will be able to efficiently and securely read the feature definitions from the registry.

#### 3. Create cloud infrastructure
### 3. Create cloud infrastructure

At this step, Feast CLI will create all necessary infrastructure for feature serving and materialization to work. What exactly gets created depends on what provider is configured to be used in `feature_store.yaml` in the Feature Repository.
Feast CLI will create all necessary infrastructure for feature serving and materialization to work. What exactly gets created depends on which provider is configured to be used in `feature_store.yaml` in the feature repository.

For example, for Local provider, it is as easy as creating a file on your local filesystem as a key-value store to serve feature data from. Local provider is most usable for local testing, no real production serving happens there.
For example, for the `local` provider, it is as easy as creating a sqlite database on disk as a key-value store to serve feature data from. The `local` provider is most usable for local testing, not production use.

A more interesting configuration is when we're configured Feast to use GCP provider and Cloud Datastore to store feature data. When you run `feast apply`, Feast will make sure you have valid credentials and create some metadata objects in the Datastore for each Feature View.

Similarly, when using AWS, Feast will make sure that resources like DynamoDB tables are created for every Feature View.
A full featured configuration is to use the `gcp` provider and Cloud Datastore to store feature data. When you run `feast apply`, Feast will initialize Datastore based on your Feast objects \(like feature views\) in order to materialize and serve features.

{% hint style="warning" %}
Since `feast deploy` \(when configured to use non-Local provider\) will create cloud infrastructure in your AWS or GCP account, it may incur some costs on your cloud bill. While we aim to design it in a way that Feast cloud resources don't cost much when not serving features, preferring "serverless" cloud services that bill per request, please refer to the specific Provider documentation to make sure there are no surprises.
Since `feast apply` \(when configured to use non-Local provider\) will create cloud infrastructure in your AWS or GCP account, it may incur some costs on your cloud bill.
{% endhint %}

4 changes: 2 additions & 2 deletions docs/feast-on-kubernetes/advanced-1/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This page applies to Feast 0.7. The content may be out of date for Feast 0.8+

### Overview

![Overview of Feast's Security Methods.](../../.gitbook/assets/untitled-25-1-%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%281%29%20%281%29.jpg)
![Overview of Feast's Security Methods.](../../.gitbook/assets/untitled-25-1-%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%281%29%20%282%29.jpg)

Feast supports the following security methods:

Expand Down Expand Up @@ -449,7 +449,7 @@ Authorization provides access control to FeatureTables and/or Features based on

#### **Authorization API/Server**

![Feast Authorization Flow](../../.gitbook/assets/rsz_untitled23%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29.jpg)
![Feast Authorization Flow](../../.gitbook/assets/rsz_untitled23%20%282%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29.jpg)

Feast delegates Authorization grants to an external Authorization Server that implements the [Authorization Open API specification](https://github.com/feast-dev/feast/blob/master/common/src/main/resources/api.yaml).

Expand Down
4 changes: 1 addition & 3 deletions docs/feast-on-kubernetes/concepts/architecture.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# Architecture



![](../../.gitbook/assets/image%20%286%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%282%29%20%281%29%20%283%29.png)
![](../../.gitbook/assets/image%20%286%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%283%29%20%282%29%20%281%29%20%282%29.png)

## Sequence description

Expand Down
2 changes: 1 addition & 1 deletion docs/feast-on-kubernetes/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

### Concept Hierarchy

![](../../.gitbook/assets/image%20%284%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%281%29.png)
![](../../.gitbook/assets/image%20%284%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%282%29%20%281%29.png)

Feast contains the following core concepts:

Expand Down
8 changes: 3 additions & 5 deletions docs/feast-on-kubernetes/concepts/stores.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Stores

In Feast, a store is a database that is populated with feature data that will ultimately be served to models.
In Feast, a store is a database that is populated with feature data that will ultimately be served to models.

### Offline \(Historical\) Store
## Offline \(Historical\) Store

The offline store maintains historical copies of feature values. These features are grouped and stored in feature tables. During retrieval of historical data, features are queries from these feature tables in order to produce training datasets.

### Online Store
## Online Store

The online store maintains only the latest values for a specific feature.

Expand All @@ -18,5 +18,3 @@ The online store maintains only the latest values for a specific feature.
Feast only supports a single online store in production
{% endhint %}



Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

In order to retrieve features for both training and serving, Feast requires data being ingested into its offline and online stores.


Users are expected to already have either a batch or stream source with data stored in it, ready to be ingested into Feast. Once a feature table \(with the corresponding sources\) has been registered with Feast, it is possible to load data from this source into stores.

The following depicts an example ingestion flow from a data source to the online store.

### Batch Source to Online Store
## Batch Source to Online Store

```python
from feast import Client
Expand All @@ -26,7 +25,7 @@ client.start_offline_to_online_ingestion(
)
```

### Stream Source to Online Store
## Stream Source to Online Store

```python
from feast import Client
Expand All @@ -39,13 +38,13 @@ driver_ft = client.get_feature_table("driver_trips")
client.start_stream_to_online_ingestion(driver_ft)
```

### Batch Source to Offline Store
## Batch Source to Offline Store

{% hint style="danger" %}
Not supported in Feast 0.8
{% endhint %}

### Stream Source to Offline Store
## Stream Source to Offline Store

{% hint style="danger" %}
Not supported in Feast 0.8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ In the example below there are two tables \(or dataframes\):

The user would like to have the driver features joined onto the entity dataframe to produce a training dataset that contains both the target \(trip\_completed\) and features \(average\_daily\_rides, maximum\_daily\_rides, rating\). This dataset will then be used to train their model.

![](../../.gitbook/assets/point_in_time_join%20%281%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%281%29.png)
![](../../.gitbook/assets/point_in_time_join%20%281%29%20%282%29%20%282%29%20%283%29%20%283%29%20%283%29%20%283%29%20%282%29.png)

Feast is able to intelligently join feature data with different timestamps to a single entity dataframe. It does this through a point-in-time join as follows:

Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ entity_df.head()
```
{% endcode %}

![](.gitbook/assets/feast-landing-page-blog-post-page-5%20%281%29%20%281%29%20%281%29%20%282%29.png)
![](.gitbook/assets/feast-landing-page-blog-post-page-5%20%281%29%20%281%29%20%281%29%20%282%29%20%282%29%20%283%29.png)

This DataFrame represents the entity keys and timestamps that we want feature values for. We can pass this Entity DataFrame into Feast, and Feast will fetch point-in-time correct features for each row:

Expand Down

0 comments on commit 1c5e2aa

Please sign in to comment.