-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Refactor] Stream Reader and Write Generics #7465
[Refactor] Stream Reader and Write Generics #7465
Conversation
Gradle Check (Jenkins) Run Completed with:
|
@saratvemulapalli, Foundation refactor to decouple GeoPoint and Joda so StreamInput and StreamOutput can be refactored to core library. I'll do the same for OpenSearchException in a follow-up after pulling up the primitive write / read methods to the BaseStreamInput BaseStreamOutput classes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the downstream plugins impact? And are you going to do all the types similar to geo point? Should we do a single pass merge on that to minimize that?
BWC tests cover the downstream impact to streamable. I can't speak, however, to how every other downstream plugin overrides internal APIs. Just like ImmutableOpenMap, they'll find out what needs to be changed when the plugin is built off the new nightly.
No. We inherited the read/writeGeneric from elastic and it is a terrible design that needs to die. We just have to carry it for BWC. I'll be deprecating this in a follow-up. I'll also (as the description states) be moving the primitives to the new base classes.
I'll be doing the primitives in a single pass PR, yes. |
libs/core/src/main/java/org/opensearch/core/common/io/stream/BaseWriteable.java
Outdated
Show resolved
Hide resolved
libs/core/src/main/java/org/opensearch/core/common/io/stream/BaseWriteable.java
Outdated
Show resolved
Hide resolved
libs/core/src/main/java/org/opensearch/core/common/io/stream/BaseWriteable.java
Outdated
Show resolved
Hide resolved
libs/core/src/main/java/org/opensearch/core/common/io/stream/BaseWriteable.java
Outdated
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
@@ Coverage Diff @@
## main #7465 +/- ##
============================================
+ Coverage 70.47% 70.60% +0.12%
- Complexity 59692 59774 +82
============================================
Files 4893 4896 +3
Lines 286748 286779 +31
Branches 41327 41331 +4
============================================
+ Hits 202091 202485 +394
+ Misses 68060 67576 -484
- Partials 16597 16718 +121
|
...les/geo/src/main/java/org/opensearch/geo/search/aggregations/bucket/geogrid/BaseGeoGrid.java
Outdated
Show resolved
Hide resolved
...in/java/org/opensearch/geo/search/aggregations/bucket/geogrid/InternalGeoHashGridBucket.java
Outdated
Show resolved
Hide resolved
...in/java/org/opensearch/geo/search/aggregations/bucket/geogrid/InternalGeoTileGridBucket.java
Outdated
Show resolved
Hide resolved
test/framework/src/main/java/org/opensearch/test/OpenSearchTestCase.java
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
EC2 connection failure.. refired gradle check |
Gradle Check (Jenkins) Run Completed with:
|
oye.. caught in a version rip current. Rebased. |
StreamInput and StreamOutput provide the core functionality for marshalling / unmarshalling objects over the transport wire. The class is intended to be generic but it is tightly coupled to the types defined in the server module. Because of this tight coupling, the classes cannot be refactored to the core library, thus all new types are required to be hard coded in the server module. To decouple this logic and make it more generic across opensearch modules and plugins, this commit introduces a reader and writer registry in a new BaseWriteable interface. The StreamInput and StreamOutput also now inherits from new BaseStreamInput and BaseStreamOutput classes, respectively, located in the core library. This will be the new home for streaming primitives in a follow up commit. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
b75c882
to
f71b607
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Love it :-)
Gradle Check (Jenkins) Run Completed with:
|
* | ||
* @opensearch.internal | ||
*/ | ||
public abstract class BaseStreamOutput extends OutputStream {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar to another PR where we're doing quite a bit of Base*
, should this have been called OutputStream extends java.io.OutputStream
, or at least BaseOutputStream extends ...
to match the order of StreamOutput and OutputStream?
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-7465-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 392325705eca9afe54d3ed38063b1ee7e0263fa2
# Push it to GitHub
git push --set-upstream origin backport/backport-7465-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x Then, create a pull request where the |
StreamInput and StreamOutput provide the core functionality for marshalling / unmarshalling objects over the transport wire. The class is intended to be generic but it is tightly coupled to the types defined in the server module. Because of this tight coupling, the classes cannot be refactored to the core library, thus all new types are required to be hard coded in the server module. To decouple this logic and make it more generic across opensearch modules and plugins, this commit introduces a reader and writer registry in a new BaseWriteable interface. The StreamInput and StreamOutput also now inherits from new BaseStreamInput and BaseStreamOutput classes, respectively, located in the core library. This will be the new home for streaming primitives in a follow up commit. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
…8457) * [Refactor] Stream Reader and Write Generics (#7465) StreamInput and StreamOutput provide the core functionality for marshalling / unmarshalling objects over the transport wire. The class is intended to be generic but it is tightly coupled to the types defined in the server module. Because of this tight coupling, the classes cannot be refactored to the core library, thus all new types are required to be hard coded in the server module. To decouple this logic and make it more generic across opensearch modules and plugins, this commit introduces a reader and writer registry in a new BaseWriteable interface. The StreamInput and StreamOutput also now inherits from new BaseStreamInput and BaseStreamOutput classes, respectively, located in the core library. This will be the new home for streaming primitives in a follow up commit. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * changelog Signed-off-by: Nicholas Walter Knize <nknize@apache.org> --------- Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
StreamInput and StreamOutput provide the core functionality for marshalling / unmarshalling objects over the transport wire. The class is intended to be generic but it is tightly coupled to the types defined in the server module. Because of this tight coupling, the classes cannot be refactored to the core library, thus all new types are required to be hard coded in the server module. To decouple this logic and make it more generic across opensearch modules and plugins, this commit introduces a reader and writer registry in a new BaseWriteable interface. The StreamInput and StreamOutput also now inherits from new BaseStreamInput and BaseStreamOutput classes, respectively, located in the core library. This will be the new home for streaming primitives in a follow up commit. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
StreamInput and StreamOutput provide the core functionality for marshalling / unmarshalling objects over the transport wire. The class is intended to be generic but it is tightly coupled to the types defined in the server module. Because of this tight coupling, the classes cannot be refactored to the core library, thus all new types are required to be hard coded in the server module.
To decouple this logic and make it more generic across opensearch modules and plugins, this commit introduces a reader and writer registry in a new BaseWriteable interface. The StreamInput and StreamOutput also now inherits from new BaseStreamInput and BaseStreamOutput classes, respectively, located in the core library. This will be the new home for streaming primitives in a follow up commit.
relates #5910