-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility #9262
Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility #9262
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
4a22989
to
ec326ae
Compare
libs/core/src/main/java/org/opensearch/core/compress/CompressorRegistry.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM to me!
Gradle Check (Jenkins) Run Completed with:
|
These test failures are unreal. |
Gradle Check (Jenkins) Run Completed with:
|
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9262-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c6b67e1b938b5e34806216af5efef32954554bf3
# Push it to GitHub
git push --set-upstream origin backport/backport-9262-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x Then, create a pull request where the |
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
… extensibility (#9262) (#9367) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
### Description Reacting to changes from the backport of opensearch-project/OpenSearch#9262. This reason the main branch of Security is not encountering these errors is because the affected class was removed in #2900 and the change has not been backported yet. @willyborankin are you planning to backport #2900? * Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation) Maintenance ### Check List - [ ] New functionality includes testing - [ ] New functionality has been documented - [ ] Commits are signed per the DCO using --signoff By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). Signed-off-by: Craig Perkins <cwperx@amazon.com>
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Kiran Reddy <kkreddy@amazon.com>
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
This PR refactors the
CompressorFactory
static singleton class andCompressorType
enum to a formalCompressorRegistry
and enables downstream implementations to register their own compression implementations through a newCompressorProvider
interface for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields.**Note the following implementation details and plans **
:libs:opensearch-compress
library - the purpose of this library is much like the:libs:opensearch-x-content
library, to provide supplemental compression implementations that are optional for opensearch API users. This design choice is for one reason: keep the core API library lean. Note this does not mean:server
is breaking because the zstd classes are not in the core library. It also does not mean downstreams HAVE to take a dependency on:opensearch-compress
(only if they want to use zstd as an option).NoneCompressor
in the core library. This doesn't mean server will break.:server
module will still use Deflate as the default store compression as it will take a dependency on :opensearch-core and :opensearch-compress. Downstream implementations of the opensearch library (e.g., such as :server) can optionally take a dependency on theopensearch-compress
library if they want to leverage Deflate and Zstd. The purpose, again, is to keep core library a minimal API implementation of the distributed lucene API.relates #5910
relates #8110
relates #2447