Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable dynamic embedding models #1060 #1063

Merged
merged 9 commits into from
Jan 4, 2025

Conversation

e7217
Copy link
Contributor

@e7217 e7217 commented Dec 19, 2024

This PR is related to issue #1060 .
I have tried to make it easier for users to apply embedding models that are not included in the embedding models list.

Below is how to apply the embedding model in the YAML file:

vectordb:
- name: autorag_test
  db_type: milvus
  embedding_model: 
  - type: huggingface
    model_name: intfloat/multilingual-e5-large-instruct
  ...

or

vectordb:
- name: autorag_test
  db_type: milvus
  embedding_model: 
  - type: openai
    model_name: text-embedding-3-small
  ...

However, the previous method can still be used:

vectordb:
- name: autorag_test
  db_type: milvus
  embedding_model: openai_embed_3_small
  ...

I have checked that this change does not impact the existing functionality.
I appreciate your review. Thank you

@vkehfdl1 vkehfdl1 self-requested a review January 4, 2025 06:38
Copy link
Contributor

@vkehfdl1 vkehfdl1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@e7217 Thanks for the contribution.
I changed a bit and add documentation.
Overall it is great feature for everyone.
We are late for the review and sorry for that.
Happy new year!!

@vkehfdl1 vkehfdl1 enabled auto-merge (squash) January 4, 2025 07:25
@vkehfdl1 vkehfdl1 merged commit 1f093d3 into Marker-Inc-Korea:main Jan 4, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Proposal to Allow Dynamic Use of Additional Embedding Models
2 participants