Do not require an object type for objects with enabled: false

[jsoriano]

What does this PR do?

Removes requirement of having an object_type for objects with enabled: false.

Why is it important?

Objects with enabled: false disable any parsing of their children elements, they are not indexed and only available on the _source. For these objects it should be possible to don't provide any mapping for subobjects.

For objects with enabled: false and object_type, Fleet will still create a mapping for the matching children objects, and Elasticsearch will still enforce it.

Checklist

• I have added test packages to test/packages that prove my change is effective.
• I have added an entry in spec/changelog.yml.

Related issues

• Relates to [Validation] Detect incorrectly defined dynamic mappings #624

[mrodm]

As it is, it would still be allowed this kind of field definitions:

         - name: user_provided_metadata
           type: object
           object_type: keyword
           enabled: false

Probably it is required to add these other conditions ?
Not tested with any other examples

Suggested change

	oneOf:
	- required:
	- object_type
	- properties:
	enabled:
	const: false
	required:
	- enabled
	oneOf:
	- required:
	- object_type
	properties:
	enabled:
	const: true
	- properties:
	enabled:
	const: false
	required:
	- enabled
	not:
	required:
	- object_type

With the example above, it would raise this error:

1. file "../../../../test/packages/good_v3/elasticsearch/transform/metadata_united/fields/fields.yml" is invalid: field 1.fields.1.fields.24.enabled: 1.fields.1.fields.24.enabled does not match: true

And for this mapping

    - name: user_provided_metadata
      type: object
      enabled: true

it would fail with this error:

1. file "../../../../test/packages/good_v3/elasticsearch/transform/metadata_united/fields/fields.yml" is invalid: field 1.fields.1.fields.24.enabled: 1.fields.1.fields.24.enabled does not match: false

[jsoriano]

As it is, it would still be allowed this kind of field definitions:

         - name: user_provided_metadata
           type: object
           object_type: keyword
           enabled: false

We have to permit this, at least for 3.0.0-3.0.2, to avoid making a breaking change. There are some packages, at least cloud_security_posture using this now. I opted with the option of being more permissive.

If we consider that is problematic to allow this (it is as we have seen), I could try to add logic to disallow this starting on 3.0.3.

• name: user_provided_metadata
  type: object
  enabled: true

This mapping should not be allowed in any case and indeed it is with the current code, I will take a look and add a test for this.

[jsoriano]

Thanks @mrodm, I think your proposal works well. I have added a JSON patch to keep things as they are before 3.0.2, so we only disallow having an object_type with enabled: false starting on 3.0.3, and we don't break current cloud_security posture package.

[mrodm]

We have to permit this, at least for 3.0.0-3.0.2, to avoid making a breaking change. There are some packages, at least cloud_security_posture using this now. I opted with the option of being more permissive.

Ok, right we need to avoid breaking changes 👍

This mapping should not be allowed in any case and indeed it is with the current code, I will take a look and add a test for this.

Thanks!

[mrodm]

We have to permit this, at least for 3.0.0-3.0.2, to avoid making a breaking change. There are some packages, at least cloud_security_posture using this now. I opted with the option of being more permissive.

Ok, right we need to avoid breaking changes 👍

This mapping should not be allowed in any case and indeed it is with the current code, I will take a look and add a test for this.

Thanks!

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 10a4adb

History

• 💚 Build #441 succeeded cb9c1c2

cc @jsoriano

[mrodm]

LGTM!