[Backport 2.x] Add FlatObject FieldMapper #7052
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![opensearch-trigger-bot[bot]](https://avatars.githubusercontent.com/u/80134844?s=60&v=4){kind=small}
* Add FlatObject FieldMapper Signed-off-by: Mingshi Liu <mingshl@amazon.com> * resolve import package for HttpHost Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Dynamic Create FlatObjectFieldType for dotpath field Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Rename flat-object to flat_object and fix CI tests Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Organized package Signed-off-by: Mingshi Liu <mingshl@amazon.com> * resolved compile error Signed-off-by: Mingshi Liu <mingshl@amazon.com> * organize package Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Add integration tests and remove benchmark Signed-off-by: Mingshi Liu <mingshl@amazon.com> * fix IT tests Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Skip IT tests before 2.7.0 Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Revert "Skip IT tests before 2.7.0" Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Add more IT tests for supported queries Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Removed license head and add tests for wildcard query Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Add tests for array, nested-arrary, number and float Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Upgrade FlatObjectFieldMapperTests to MapperTestCase - Upgrading `FlatObjectFieldMapperTests` from `MapperServiceTestCase` to `MapperTestCase`. The `MapperTestCase` explicitly forces us to: - Test parameter updates (empty now) - Explicitly specify if the field supports Meta and Boost (if not, relevant tests are automatically skipped) - Test also the substring fields - Add new test `testMapperServiceHasParser` to verify the new `flat_object` field mapper is present in mapper service registry - Remove duplicated test and assertions methods - Removed `testExistsQueryDocValuesDisabledWithNorms` as this test was not adding much. We shall reintroduce similar test later if we decide that we want to support ExistsQuery and what to do if DocValue are disabled and Norms enabled. Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> * Add exist query in FlatObjectFieldMapperTests and FlatObjectFieldDataTests Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Add IT tests for painless query in testDocValues Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Add filter script (Painless) test for flat_object While it is not possible to use flat_object field in scripting filter context to access doc values (like `doc[<flat_object>.<field_x>]`) it is possible to call `doc[<flat_object>].size()` to get number of fields inside the flat_object field. - Reorganize flat_object yaml tests into sections: - Mappings - Supported - Unsupported - Scripting (Painless) yamlRest tests need to go into lang-painless module Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> * Removed Normalizer Signed-off-by: Mingshi Liu <mingshl@amazon.com> * removed unused codes Signed-off-by: Mingshi Liu <mingshl@amazon.com> * Remove non-relevant Javadoc from DynamicKeyFieldMapper Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> * Improve flat_object scripting test Make it more obvious what the `doc[<flat_field>].size()` number represents. Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> * Add test for mapping parameters Mapping parameters are not allowed in the initial version. This commit adds a test to demonstrate that trying to specify index/search analyzer for the flat_object field will fail. Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> * remove IndexAnalyzer from Builder Signed-off-by: Mingshi Liu <mingshl@amazon.com> * remove IndexAnalyzer from Builder Signed-off-by: Mingshi Liu <mingshl@amazon.com> --------- Signed-off-by: Mingshi Liu <mingshl@amazon.com> Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io> Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io> (cherry picked from commit 75bb3ef) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot
bot
requested review from
reta,
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
gbbafna,
setiah,
kartg,
kotwanikunal,
mch2,
nknize,
owaiskazi19,
Rishikesh1159,
ryanbogan,
saratvemulapalli,
shwetathareja,
dreamer-89,
tlfeng,
VachaShah and
xuezhou25
as code owners
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## 2.x #7052 +/- ##
============================================
- Coverage 70.36% 70.28% -0.09%
+ Complexity 59577 59494 -83
============================================
Files 4822 4824 +2
Lines 285929 286277 +348
Branches 41558 41621 +63
============================================
+ Hits 201200 201207 +7
- Misses 67968 68249 +281
- Partials 16761 16821 +60
... and 456 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
tlfeng
added
backport
tlfeng
approved these changes
Apr 7, 2023
There was a problem hiding this comment.
A reminder that a separate PR needed to change the condition for running the rest api test, for main branch, and then backport that to 2.x branch.
"Flat-object fields from within the scripting":
- skip:
version: " - 2.6.99"
reason: "flat_object is introduced in 2.7.0"
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 75bb3ef from #6507.