Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request includes several changes to the
BGEEmbeddings
class in thepython/sparknlp
module and various Scala classes in thecom.johnsnowlabs.ml.ai
package. The most significant changes involve adding a new property to handle CLS token usage, modifying methods to support this new property, and cleaning up code formatting.Enhancements to
BGEEmbeddings
:HasClsTokenProperties
to theBGEEmbeddings
class and updated the__init__
method to include theuseCLSToken
parameter. (python/sparknlp/annotator/embeddings/bge_embeddings.py
) [1] [2] [3]pretrained
method to use a new default model namebge_small_en_v1.5
. (python/sparknlp/annotator/embeddings/bge_embeddings.py
)New property class:
HasClsTokenProperties
class with methods to set and get theuseCLSToken
parameter. (python/sparknlp/common/properties.py
)Scala class updates:
BGE
class to support theuseCLSToken
parameter in methods for sentence embedding computation, including handling different pooling strategies based on this parameter. (src/main/scala/com/johnsnowlabs/ml/ai/BGE.scala
) [1] [2] [3] [4]Albert
andBart
classes by removing unnecessary blank lines and improving code readability. (src/main/scala/com/johnsnowlabs/ml/ai/Albert.scala
,src/main/scala/com/johnsnowlabs/ml/ai/Bart.scala
) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]These changes enhance the functionality and maintainability of the codebase by introducing new features and improving code clarity.