Skip to content

Commit

Permalink
Add strict_allow_templates dynamic mapping option (#14555)
Browse files Browse the repository at this point in the history
* The dynamic mapping parameter supports strict_allow_templates

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify skip version in yml test file

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Refactor some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Keep the old methods

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* change public to private

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Do not override toString method for Dynamic

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code and modify the changelog

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
  • Loading branch information
gaobinlong authored Jul 11, 2024
1 parent 0848525 commit 6b8b3ef
Show file tree
Hide file tree
Showing 8 changed files with 688 additions and 57 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- [Workload Management] Add QueryGroup schema ([13669](https://github.com/opensearch-project/OpenSearch/pull/13669))
- Add batching supported processor base type AbstractBatchingProcessor ([#14554](https://github.com/opensearch-project/OpenSearch/pull/14554))
- Fix race condition while parsing derived fields from search definition ([14445](https://github.com/opensearch-project/OpenSearch/pull/14445))
- Add `strict_allow_templates` dynamic mapping option ([#14555](https://github.com/opensearch-project/OpenSearch/pull/14555))
- Add allowlist setting for ingest-common and search-pipeline-common processors ([#14439](https://github.com/opensearch-project/OpenSearch/issues/14439))
- Create SystemIndexRegistry with helper method matchesSystemIndex ([#14415](https://github.com/opensearch-project/OpenSearch/pull/14415))
- Print reason why parent task was cancelled ([#14604](https://github.com/opensearch-project/OpenSearch/issues/14604))
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
---
"Index documents with setting dynamic parameter to strict_allow_templates in the mapping of the index":
- skip:
version: " - 2.99.99"
reason: "introduced in 3.0.0"

- do:
indices.create:
index: test_1
body:
mappings:
dynamic: strict_allow_templates
dynamic_templates: [
{
strings: {
"match": "stringField*",
"match_mapping_type": "string",
"mapping": {
"type": "keyword"
}
}
},
{
object: {
"match": "objectField*",
"match_mapping_type": "object",
"mapping": {
"type": "object",
"properties": {
"bar1": {
"type": "keyword"
},
"bar2": {
"type": "text"
}
}
}
}
},
{
boolean: {
"match": "booleanField*",
"match_mapping_type": "boolean",
"mapping": {
"type": "boolean"
}
}
},
{
double: {
"match": "doubleField*",
"match_mapping_type": "double",
"mapping": {
"type": "double"
}
}
},
{
long: {
"match": "longField*",
"match_mapping_type": "long",
"mapping": {
"type": "long"
}
}
},
{
array: {
"match": "arrayField*",
"mapping": {
"type": "keyword"
}
}
},
{
date: {
"match": "dateField*",
"match_mapping_type": "date",
"mapping": {
"type": "date"
}
}
}
]
properties:
test1:
type: text

- do:
catch: /mapping set to strict_allow_templates, dynamic introduction of \[test2\] within \[\_doc\] is not allowed/
index:
index: test_1
id: 1
body: {
stringField: bar,
objectField: {
bar1: "bar1",
bar2: "bar2"
},
test1: test1,
test2: test2
}

- do:
index:
index: test_1
id: 1
body: {
stringField: bar,
objectField: {
bar1: "bar1",
bar2: "bar2"
},
booleanField: true,
doubleField: 1.0,
longField: 100,
arrayField: ["1","2"],
dateField: "2024-06-25T05:11:51.243Z",
test1: test1
}

- do:
get:
index: test_1
id: 1
- match: { _source: {
stringField: bar,
objectField: {
bar1: "bar1",
bar2: "bar2"
},
booleanField: true,
doubleField: 1.0,
longField: 100,
arrayField: [ "1","2" ],
dateField: "2024-06-25T05:11:51.243Z",
test1: test1
}
}

- do:
indices.get_mapping: {
index: test_1
}

- match: {test_1.mappings.dynamic: strict_allow_templates}
- match: {test_1.mappings.properties.stringField.type: keyword}
- match: {test_1.mappings.properties.objectField.properties.bar1.type: keyword}
- match: {test_1.mappings.properties.objectField.properties.bar2.type: text}
- match: {test_1.mappings.properties.booleanField.type: boolean}
- match: {test_1.mappings.properties.doubleField.type: double}
- match: {test_1.mappings.properties.longField.type: long}
- match: {test_1.mappings.properties.arrayField.type: keyword}
- match: {test_1.mappings.properties.dateField.type: date}
- match: {test_1.mappings.properties.test1.type: text}
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,34 @@ setup:
indices.get_mapping: {}

- match: {test_index1.mappings.properties.text.type: text}

---
"post a mapping with setting dynamic to strict_allow_templates":
- skip:
version: " - 2.99.99"
reason: "introduced in 3.0.0"
- do:
indices.put_mapping:
index: test_index1
body:
dynamic: strict_allow_templates
dynamic_templates: [
{
strings: {
"match": "foo*",
"match_mapping_type": "string",
"mapping": {
"type": "keyword"
}
}
}
]
properties:
test1:
type: text

- do:
indices.get_mapping: {}

- match: {test_index1.mappings.dynamic: strict_allow_templates}
- match: {test_index1.mappings.properties.test1.type: text}
Loading

0 comments on commit 6b8b3ef

Please sign in to comment.