-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility #9262
Merged
andrross
merged 8 commits into
opensearch-project:main
from
nknize:compressorSPIExtensions
Aug 15, 2023
Merged
Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility #9262
andrross
merged 8 commits into
opensearch-project:main
from
nknize:compressorSPIExtensions
Aug 15, 2023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nknize
added
enhancement
Enhancement or improvement to existing feature or request
pending backport
Identifies an issue or PR that still needs to be backported
skip-changelog
v2.10.0
labels
Aug 11, 2023
nknize
requested review from
reta,
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
gbbafna,
setiah,
kartg,
kotwanikunal,
mch2,
owaiskazi19,
Rishikesh1159,
ryanbogan,
saratvemulapalli,
shwetathareja,
dreamer-89,
tlfeng,
VachaShah,
dbwiddis,
sachinpkale and
sohami
as code owners
August 11, 2023 13:58
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:
|
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
nknize
force-pushed
the
compressorSPIExtensions
branch
from
August 14, 2023 19:20
4a22989
to
ec326ae
Compare
reta
reviewed
Aug 14, 2023
libs/core/src/main/java/org/opensearch/core/compress/CompressorRegistry.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
reta
approved these changes
Aug 14, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM to me!
Gradle Check (Jenkins) Run Completed with:
|
These test failures are unreal. |
Gradle Check (Jenkins) Run Completed with:
|
andrross
approved these changes
Aug 15, 2023
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9262-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c6b67e1b938b5e34806216af5efef32954554bf3
# Push it to GitHub
git push --set-upstream origin backport/backport-9262-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x Then, create a pull request where the |
nknize
added a commit
to nknize/OpenSearch
that referenced
this pull request
Aug 15, 2023
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
nknize
added a commit
to nknize/OpenSearch
that referenced
this pull request
Aug 15, 2023
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
nknize
added a commit
to nknize/OpenSearch
that referenced
this pull request
Aug 17, 2023
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
nknize
added a commit
to nknize/OpenSearch
that referenced
this pull request
Aug 21, 2023
… extensibility (opensearch-project#9262) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
nknize
added a commit
that referenced
this pull request
Aug 21, 2023
… extensibility (#9262) (#9367) This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit c6b67e1)
3 tasks
cwperks
added a commit
to opensearch-project/security
that referenced
this pull request
Aug 21, 2023
### Description Reacting to changes from the backport of opensearch-project/OpenSearch#9262. This reason the main branch of Security is not encountering these errors is because the affected class was removed in #2900 and the change has not been backported yet. @willyborankin are you planning to backport #2900? * Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation) Maintenance ### Check List - [ ] New functionality includes testing - [ ] New functionality has been documented - [ ] Commits are signed per the DCO using --signoff By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). Signed-off-by: Craig Perkins <cwperx@amazon.com>
kkmr
pushed a commit
to kkmr/OpenSearch
that referenced
this pull request
Aug 28, 2023
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Kiran Reddy <kkreddy@amazon.com>
This was referenced Sep 1, 2023
kaushalmahi12
pushed a commit
to kaushalmahi12/OpenSearch
that referenced
this pull request
Sep 12, 2023
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
brusic
pushed a commit
to brusic/OpenSearch
that referenced
this pull request
Sep 25, 2023
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
shiv0408
pushed a commit
to Gaurav614/OpenSearch
that referenced
this pull request
Apr 25, 2024
… extensibility (opensearch-project#9262) * Refactor Compressors from CompressorFactory to CompressorRegistry for extensibility This commit refactors the CompressorFactory static singleton class and CompressorType enum to a formal CompressorRegistry and enables downstream implementations to register their own compression implementations for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix missing javadoc and relocate the common compress to compress package Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix deprecated case Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix typo in compress build.gradle Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR cleanup Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * PR changes Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * update with initial annotations Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * remove NONE singleton in CompressorRegistry and remove static block init Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
backport 2.x
Backport to 2.x branch
backport-failed
enhancement
Enhancement or improvement to existing feature or request
pending backport
Identifies an issue or PR that still needs to be backported
skip-changelog
v2.10.0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors the
CompressorFactory
static singleton class andCompressorType
enum to a formalCompressorRegistry
and enables downstream implementations to register their own compression implementations through a newCompressorProvider
interface for use in compressing Blob stores and MediaType data. This is different from Lucene's Codec compression extension points which expose different compression implementations for Lucene's Stored Fields.**Note the following implementation details and plans **
:libs:opensearch-compress
library - the purpose of this library is much like the:libs:opensearch-x-content
library, to provide supplemental compression implementations that are optional for opensearch API users. This design choice is for one reason: keep the core API library lean. Note this does not mean:server
is breaking because the zstd classes are not in the core library. It also does not mean downstreams HAVE to take a dependency on:opensearch-compress
(only if they want to use zstd as an option).NoneCompressor
in the core library. This doesn't mean server will break.:server
module will still use Deflate as the default store compression as it will take a dependency on :opensearch-core and :opensearch-compress. Downstream implementations of the opensearch library (e.g., such as :server) can optionally take a dependency on theopensearch-compress
library if they want to leverage Deflate and Zstd. The purpose, again, is to keep core library a minimal API implementation of the distributed lucene API.relates #5910
relates #8110
relates #2447