Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: Add embedding models support #418

Merged
merged 6 commits into from
Sep 5, 2023

Conversation

aresnow1
Copy link
Contributor

@aresnow1 aresnow1 commented Sep 1, 2023

Add embedding models support.

  • Base support for model load and encode.
  • Refactor some APIs to support launching and serving embedding models.
  • Add top 5 or top 10 models in hf leadboard.

@XprobeBot XprobeBot added this to the v0.2.1 milestone Sep 1, 2023
@aresnow1 aresnow1 marked this pull request as ready for review September 2, 2023 09:00
@aresnow1
Copy link
Contributor Author

aresnow1 commented Sep 2, 2023

Sample code:

In [7]: client = Client("http://localhost:9997")

In [8]: uid = client.launch_model(model_name="gte-base", model_type="embedding")

In [9]: model = client.get_model(model_uid=uid)

In [10]: model.create_embedding("write a poem.")
Out[10]:
{'object': 'list',
 'model': '3ef99480-496f-11ee-9009-c2c8e4cad3f6',
 'data': [{'index': 0,
   'object': 'embedding',
   'embedding': [-0.003699747147038579,
    0.019329899922013283,
    ...,
    -0.05098249390721321,
    -0.001861078548245132]}],
 'usage': {'prompt_tokens': 37, 'total_tokens': 37}}

xinference/model/core.py Outdated Show resolved Hide resolved
xinference/model/embedding/core.py Outdated Show resolved Hide resolved
@XprobeBot XprobeBot modified the milestones: v0.2.1, v0.3.1 Sep 5, 2023
@UranusSeven UranusSeven merged commit 090d619 into xorbitsai:main Sep 5, 2023
@aresnow1 aresnow1 deleted the feat/embedding branch September 5, 2023 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants