Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/553 release candidate #14511

Merged
merged 10 commits into from
Jan 30, 2025
Merged

Conversation

maziyarpanahi
Copy link
Member

@maziyarpanahi maziyarpanahi commented Jan 29, 2025

This pull request includes several changes to the BGEEmbeddings class in the python/sparknlp module and various Scala classes in the com.johnsnowlabs.ml.ai package. The most significant changes involve adding a new property to handle CLS token usage, modifying methods to support this new property, and cleaning up code formatting.

Enhancements to BGEEmbeddings:

  • Added HasClsTokenProperties to the BGEEmbeddings class and updated the __init__ method to include the useCLSToken parameter. (python/sparknlp/annotator/embeddings/bge_embeddings.py) [1] [2] [3]
  • Updated the pretrained method to use a new default model name bge_small_en_v1.5. (python/sparknlp/annotator/embeddings/bge_embeddings.py)

New property class:

  • Introduced HasClsTokenProperties class with methods to set and get the useCLSToken parameter. (python/sparknlp/common/properties.py)

Scala class updates:

  • Modified the BGE class to support the useCLSToken parameter in methods for sentence embedding computation, including handling different pooling strategies based on this parameter. (src/main/scala/com/johnsnowlabs/ml/ai/BGE.scala) [1] [2] [3] [4]
  • Cleaned up code formatting in Albert and Bart classes by removing unnecessary blank lines and improving code readability. (src/main/scala/com/johnsnowlabs/ml/ai/Albert.scala, src/main/scala/com/johnsnowlabs/ml/ai/Bart.scala) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]

These changes enhance the functionality and maintainability of the codebase by introducing new features and improving code clarity.

@maziyarpanahi maziyarpanahi merged commit 7d2bed7 into master Jan 30, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants