Make mapping updates more robust. #15539
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jpountz
added
>enhancement
:Search Foundations/Mapping
| fieldMappers.put(mapper.name(), mapper); | ||
| MappedFieldType fieldType = mapper.fieldType(); | ||
| put(indexAnalyzers, fieldType.names().indexName(), fieldType.indexAnalyzer(), analysisService.defaultIndexAnalyzer()); | ||
| put(indexAnalyzers, fieldType.names().indexName(), fieldType.searchAnalyzer(), analysisService.defaultSearchAnalyzer()); |
There was a problem hiding this comment.
Typo here? indexAnalyzers -> searchAnalyzers? And indexAnalyzers -> searchQuoteAnalyzers on the next line?
There was a problem hiding this comment.
Good catch. This is concerning that no tests caught this mistake, I'll add one.
|
I discussed with @jpountz and I agree now making mappings completely immutable is more important here. We should separately drive the simplification of types (ie type removal/turning them into just a simple set-of-fields-selector). +1 to this PR. |
jpountz
force-pushed
the
fix/immutable_document_mapper
branch
from
6ff0215 to
6bc7f97
Compare
|
@rjernst I rebased with master and pushed a new commit to address the problem you found with index analyzers. |
|
LGTM |
This changes a couple of things: Mappings are truly immutable. Before, each field mapper stored a MappedFieldTypeReference that was shared across fields that have the same name across types. This means that a mapping update could have the side-effect of changing the field type in other types when updateAllTypes is true. This works differently now: after a mapping update, a new copy of the mappings is created in such a way that fields across different types have the same MappedFieldType. See the new Mapper.updateFieldType API which replaces MappedFieldTypeReference. DocumentMapper is now immutable and MapperService.merge has been refactored in such a way that if an exception is thrown while eg. lookup structures are being updated, then the whole mapping update will be aborted. As a consequence, FieldTypeLookup's checkCompatibility has been folded into copyAndAddAll. Synchronization was simplified: given that mappings are truly immutable, we don't need the read/write lock so that no documents can be parsed while a mapping update is being processed. Document parsing is not performed under a lock anymore, and mapping merging uses a simple synchronized block.
jpountz
force-pushed
the
fix/immutable_document_mapper
branch
from
6bc7f97 to
f535c27
Compare
jpountz
added a commit
that referenced
this pull request
Dec 23, 2015
Make mapping updates more robust.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This changes a couple of things:
Mappings are truly immutable. Before, each field mapper stored a
MappedFieldTypeReference that was shared across fields that have the same name
across types. This means that a mapping update could have the side-effect of
changing the field type in other types when updateAllTypes is true. This works
differently now: after a mapping update, a new copy of the mappings is created
in such a way that fields across different types have the same MappedFieldType.
See the new Mapper.updateFieldType API which replaces MappedFieldTypeReference.
DocumentMapper is now immutable and MapperService.merge has been refactored in
such a way that if an exception is thrown while eg. lookup structures are being
updated, then the whole mapping update will be aborted. As a consequence,
FieldTypeLookup's checkCompatibility has been folded into copyAndAddAll.
Synchronization was simplified: given that mappings are truly immutable, we
don't need the read/write lock so that no documents can be parsed while a
mapping update is being processed. Document parsing is not performed under a
lock anymore, and mapping merging uses a simple synchronized block.