Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add remote-http online store #4121

Closed
tokoko opened this issue Apr 19, 2024 · 11 comments
Closed

Add remote-http online store #4121

tokoko opened this issue Apr 19, 2024 · 11 comments
Labels
kind/feature New feature or request

Comments

@tokoko
Copy link
Collaborator

tokoko commented Apr 19, 2024

Is your feature request related to a problem? Please describe.
Users can deploy http and grpc feature servers that wrap get_online_features calls, but currently there is no way to interact with feature servers from python sdk.

Describe the solution you'd like
Add online store types remote-http and remote-grpc (or maybe just call it a remote and differentiate with a url?) that wrap network calls to the respective feature servers. This follows the same pattern as remote registry implementation that wraps calls to the registry server.

@tokoko tokoko added the kind/feature New feature or request label Apr 19, 2024
@tokoko tokoko changed the title Add remote-http and remote-grpc online stores Add remote-http online store Apr 25, 2024
@lokeshrangineni
Copy link
Contributor

I am currently working on this and will get back in case if i find any difficulty.

@lokeshrangineni
Copy link
Contributor

@franciscojavierarceo FYI

@franciscojavierarceo
Copy link
Member

franciscojavierarceo commented May 3, 2024

Some things we'll have to do:

1. Handle purely feature_views i.e., features="feature_view"
2. Handle feature references; i.e., ["feature_view:feature_1","feature_view:feature_2",...] etc.
3. Handle feature services

We'll have to serialize the data on the server to match what's in the feature registry from the FeatureView itself and fail loudly when it doesn't match, probably return a 400.

@tokoko
Copy link
Collaborator Author

tokoko commented May 3, 2024

@franciscojavierarceo I think you closed this by accident.

  • Handle purely feature_views i.e., features="feature_view"

Is this something we support? The signature for get_online_features is only feature service or a list of features -> features: Union[List[str], FeatureService]

@franciscojavierarceo
Copy link
Member

Whoops, nevermind that was based on outdated documentation I think.

@franciscojavierarceo
Copy link
Member

@tokoko I'm confused about why we would do this? I think adding a client that makes the behavior intuitive makes sense but that's basically in the Python feature server.

@tokoko
Copy link
Collaborator Author

tokoko commented May 3, 2024

Well, it's more of convenience thing, i think. They can still use the feature server directly of course, but what this additional layer would enable is writing a piece of code using store.get_online_features and then configuring how you want to resolve features with feature_store.yaml. You could change the backend from the actual online store to http feature server or a grpc one without changing anything in the code itself.

@lokeshrangineni
Copy link
Contributor

lokeshrangineni commented May 15, 2024

Few updates after started working on the issue:

  1. There is no grpc implementation of APIs in python feature server. So this issue is going to only focus on remote with http implementation. We can enhance the online store to include grpc implementation when the grpc apis available on the python feature server.
  2. Planning to implement the online store api methods of online_write_batch and online_read. Rest all life cycle methods of online_store are not having associated APIs in the feature server.

@dmartinol
Copy link
Contributor

@tokoko can you confirm the above assumptions, in order to guide @lokeshrangineni 's implementation?

@tokoko
Copy link
Collaborator Author

tokoko commented May 15, 2024

sure, sounds like a good starting point.

@lokeshrangineni
Copy link
Contributor

Created a draft PR just for initial feedback. Please note that code may change significantly depending on the feedback.
#4226

@tokoko tokoko closed this as completed Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants