Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disallow "enabled" attribute change for types in mapping update (#33566) #33933

Merged
merged 13 commits into from
Oct 1, 2018
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,8 @@ protected void doMerge(final ObjectMapper mergeWith) {

for (Mapper mergeWithMapper : mergeWith) {
Mapper mergeIntoMapper = mappers.get(mergeWithMapper.simpleName());
checkEnabledFieldChange(mergeWith, mergeWithMapper, mergeIntoMapper);

Mapper merged;
if (mergeIntoMapper == null) {
// no mapping, simply add it
Expand All @@ -470,6 +472,19 @@ protected void doMerge(final ObjectMapper mergeWith) {
}
}

private static void checkEnabledFieldChange(ObjectMapper mergeWith, Mapper mergeWithMapper, Mapper mergeIntoMapper) {
if (mergeIntoMapper instanceof ObjectMapper && mergeWithMapper instanceof ObjectMapper) {
final ObjectMapper mergeIntoObjectMapper = (ObjectMapper) mergeIntoMapper;
final ObjectMapper mergeWithObjectMapper = (ObjectMapper) mergeWithMapper;

if (mergeIntoObjectMapper.isEnabled() != mergeWithObjectMapper.isEnabled()) {
final String path = mergeWith.fullPath() + "." + mergeWithObjectMapper.simpleName() + ".enabled";

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe delete this line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, these two lines are related, no need to add a blank line, I'll remove it.

throw new MapperException("Can't update attribute for type [" + path + "] in index mapping");
}
}
}

@Override
public ObjectMapper updateFieldType(Map<String, MappedFieldType> fullNameToFieldType) {
List<Mapper> updatedMappers = null;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.index.mapper;

import com.google.common.collect.ImmutableMap;
import org.elasticsearch.common.Explicit;
import org.elasticsearch.common.joda.FormatDateTimeFormatter;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.index.mapper.FieldMapper.CopyTo;
import org.elasticsearch.index.mapper.FieldMapper.MultiFields;
import org.elasticsearch.index.mapper.TextFieldMapper.TextFieldType;
import org.elasticsearch.test.ESTestCase;
import org.junit.AfterClass;

import java.util.Map;

import static java.util.Collections.emptyMap;
import static org.elasticsearch.cluster.metadata.IndexMetaData.SETTING_VERSION_CREATED;
import static org.elasticsearch.index.mapper.ObjectMapper.Dynamic;
import static org.elasticsearch.index.mapper.ObjectMapper.Nested;
import static org.hamcrest.Matchers.notNullValue;

public class ObjectMapperMergeTests extends ESTestCase {
cbuescher marked this conversation as resolved.
Show resolved Hide resolved

private static FieldMapper barFieldMapper = createTextFieldMapper("bar");
private static FieldMapper bazFieldMapper = createTextFieldMapper("baz");

private static RootObjectMapper rootObjectMapper = createRootObjectMapper(
"type1", true, ImmutableMap.of(
"disabled", createObjectMapper("disabled", false, emptyMap()),
"foo", createObjectMapper("foo", true, ImmutableMap.of(
"bar", barFieldMapper))));

private static Explicit<Boolean> dateDetection = new Explicit<>(false, false);
private static Explicit<Boolean> numericDetection = new Explicit<>(false, false);
private static Explicit<FormatDateTimeFormatter[]> dynamicDateTimeFormatters = new Explicit<>(new FormatDateTimeFormatter[0], false);
private static Explicit<DynamicTemplate[]> dynamicTemplates = new Explicit<>(new DynamicTemplate[0], false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit suprised by these until I saw them being used in createRootObjectMapper(), maybe you could just instantiate them there locally, even if that means a few more instances. I think it makes the test more readable although this way is certainly "better" in terms of object creation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right. It would be a better balance between performances and readability. I'll do it, thank you.


@AfterClass
public static void cleanupReferences() {
cbuescher marked this conversation as resolved.
Show resolved Hide resolved
barFieldMapper = null;
bazFieldMapper = null;
rootObjectMapper = null;

dateDetection = null;
numericDetection = null;
dynamicDateTimeFormatters = null;
dynamicTemplates = null;
}

public void testMerge() {
// GIVEN an enriched mapping with "baz" new field
ObjectMapper mergeWith = createRootObjectMapper(
"type1", true, ImmutableMap.of(
"disabled", createObjectMapper("disabled", false, emptyMap()),
"foo", createObjectMapper("foo", true, ImmutableMap.of(
"bar", barFieldMapper,
"baz", bazFieldMapper))));

// WHEN merging mappings
final ObjectMapper merged = rootObjectMapper.merge(mergeWith);

// THEN "baz" new field is added to merged mapping
final ObjectMapper mergedFoo = (ObjectMapper) merged.getMapper("foo");
assertThat(mergedFoo.getMapper("bar"), notNullValue());
assertThat(mergedFoo.getMapper("baz"), notNullValue());
}

public void testMergeWhenDisablingField() {
// GIVEN a mapping with "foo" field disabled
ObjectMapper mergeWith = createRootObjectMapper(
"type1", true, ImmutableMap.of(
"disabled", createObjectMapper("disabled", false, emptyMap()),
"foo", createObjectMapper("foo", false, emptyMap())));

// WHEN merging mappings
// THEN a MapperException is thrown with an excepted message
MapperException e = expectThrows(MapperException.class, () -> rootObjectMapper.merge(mergeWith));
assertEquals("Can't update attribute for type [type1.foo.enabled] in index mapping", e.getMessage());
}

public void testMergeWhenEnablingField() {
// GIVEN a mapping with "disabled" field enabled
ObjectMapper mergeWith = createRootObjectMapper(
"type1", true, ImmutableMap.of(
"disabled", createObjectMapper("disabled", true, emptyMap()),
"foo", createObjectMapper("foo", true, ImmutableMap.of(
"bar", barFieldMapper))));

// WHEN merging mappings
// THEN a MapperException is thrown with an excepted message
MapperException e = expectThrows(MapperException.class, () -> rootObjectMapper.merge(mergeWith));
assertEquals("Can't update attribute for type [type1.disabled.enabled] in index mapping", e.getMessage());
}

private static RootObjectMapper createRootObjectMapper(String name, boolean enabled, Map<String, Mapper> mappers) {
return new RootObjectMapper(
name, enabled, null, mappers,
dynamicDateTimeFormatters, dynamicTemplates,
dateDetection, numericDetection,
Settings.EMPTY
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'm not too familiar with mapper creation (especially in tests) myself, but I saw that the RootObjectMapper.Builder already takes care of a lot of the defaults, so maybe this can be simplified to something like:

        final Settings indexSettings = Settings.builder().put(SETTING_VERSION_CREATED, Version.CURRENT).build();
        BuilderContext context = new BuilderContext(indexSettings, new ContentPath());
        RootObjectMapper rootObjectMapper = new RootObjectMapper.Builder(fieldName).enabled(enabled).build(context);
        mappers.values().stream().forEach(rootObjectMapper::putMapper);
        return rootObjectMapper;

Or maybe even the dummy settings could be a test-wide constant.

I don't think this is necesarry but removes a bit of boilerplate code (which I admit testing the mappers seems to require quite a bit).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's far more better, builders are far more expressive than lengthy constructors.

}

private static ObjectMapper createObjectMapper(String name, boolean enabled, Map<String, Mapper> mappers) {
return new ObjectMapper(name, name, enabled, Nested.NO, Dynamic.FALSE, mappers, Settings.EMPTY);
}

private static TextFieldMapper createTextFieldMapper(String name) {
final TextFieldType fieldType = new TextFieldType();
final Settings indexSettings = Settings.builder().put(SETTING_VERSION_CREATED, "1").build();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I guess it doesn't matter here too much, but normally one of the version constants from org.elasticsearch.Version is used in these cases, usually Version.CURRENT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's better as it's closer to production code.


return new TextFieldMapper(name, fieldType, fieldType, -1, null, indexSettings, MultiFields.empty(), CopyTo.empty());
}
}