Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

async support for embeddings #628

Open
simonw opened this issue Nov 13, 2024 · 1 comment
Open

async support for embeddings #628

simonw opened this issue Nov 13, 2024 · 1 comment
Labels
asyncio enhancement New feature or request

Comments

@simonw
Copy link
Owner

simonw commented Nov 13, 2024

Split from:

@simonw
Copy link
Owner Author

simonw commented Nov 22, 2024

Got o1-preview to attempt a first version of this: https://gist.github.com/simonw/47a9dd3a889001a147d5923f56a7f35c

files-to-prompt llm -e py -c | llm -m o1-preview "Implement asynchronous support for embedding models, inspired by how asynchronous support for regular models work. You'll need to build a new base class for these that shares some implementation with llm.models.EmbeddingModel - plus build a get_async_embedding_model() and _models and suchlike, plus implement async embeddings in the OpenAI example plugin using the async versio of client.embeddings.create. Output just the files that need changes plus new tests." -u

That prompt:

Implement asynchronous support for embedding models, inspired by how asynchronous support for regular models work. You'll need to build a new base class for these that shares some implementation with llm.models.EmbeddingModel - plus build a get_async_embedding_model() and _models and suchlike, plus implement async embeddings in the OpenAI example plugin using the async versio of client.embeddings.create. Output just the files that need changes plus new tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
asyncio enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant