Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.9] Add datastream.* to mappings #60592

Merged
merged 2 commits into from
Aug 4, 2020
Merged

Conversation

ruflin
Copy link
Member

@ruflin ruflin commented Aug 3, 2020

The current Elastic Agent ships data to dataset.*. We plant to rename this field to datastream.type, datastream.dataset, datastream.namespace instead. To make the template forward compatible, we add these fields to 7.9 and newer.

This will need forward porting to 7.x

@ruflin ruflin requested review from dakrone and jpountz August 3, 2020 13:06
@ruflin ruflin self-assigned this Aug 3, 2020
@ruflin ruflin added the v7.9.0 label Aug 3, 2020
"properties": {
"type": {
"type": "constant_keyword",
"value": "logs"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"value": "logs"
"value": "metrics"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:-( fixed.

@jpountz jpountz added :Data Management/Indices APIs APIs to create and manage indices and templates >enhancement labels Aug 3, 2020
@jpountz
Copy link
Contributor

jpountz commented Aug 3, 2020

I wonder if this actually requires to be merged in 7.9 (as opposed to 7.10), given that Elasticsearch must be upgraded before Agent?

@ruflin
Copy link
Member Author

ruflin commented Aug 3, 2020

Yes, as Elasticsearch will not update the template if a template is already there and we plan in 7.9 to already ship datastream.* values from the Agent.

@jpountz
Copy link
Contributor

jpountz commented Aug 3, 2020

Thanks, I had misunderstood the plan.

@ruflin ruflin changed the title Add datastream.* to mappings [7.9] Add datastream.* to mappings Aug 3, 2020
@ruflin ruflin marked this pull request as ready for review August 3, 2020 14:54
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Indices APIs)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Aug 3, 2020
@ruflin ruflin requested a review from jpountz August 3, 2020 14:54
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but @ruflin I think this needs additional labels for it being forward-ported? Usually we start with master and backport from there

@jpountz
Copy link
Contributor

jpountz commented Aug 4, 2020

I opened #60638 and will treat this PR as a backport PR.

@jpountz jpountz added backport and removed :Data Management/Indices APIs APIs to create and manage indices and templates >enhancement Team:Data Management Meta label for data/management team v7.9.0 labels Aug 4, 2020
@jpountz jpountz merged commit e099b10 into elastic:7.9 Aug 4, 2020
ruflin added a commit to ruflin/integrations that referenced this pull request Aug 4, 2020
It is planned to move from dataset.* to datastream.*. To make the path easier to 7.9 Elasticsearch (elastic/elasticsearch#60592) and the Elastic Agent already ship with the datastream.* fields. Because of this, these should also be added to the mappings of the packages.
ruflin added a commit to elastic/integrations that referenced this pull request Aug 4, 2020
It is planned to move from dataset.* to datastream.*. To make the path easier to 7.9 Elasticsearch (elastic/elasticsearch#60592) and the Elastic Agent already ship with the datastream.* fields. Because of this, these should also be added to the mappings of the packages.

The agent will ship datastream.* fields. Because of this, the import scripts can be adjusted to only use these fields.

This PR doese not rename any Golang variables. This should be done in a follow up and in sync with potential changes to the registry.
@ruflin ruflin deleted the update-mappings branch August 4, 2020 14:10
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
It is planned to move from dataset.* to datastream.*. To make the path easier to 7.9 Elasticsearch (elastic/elasticsearch#60592) and the Elastic Agent already ship with the datastream.* fields. Because of this, these should also be added to the mappings of the packages.

The agent will ship datastream.* fields. Because of this, the import scripts can be adjusted to only use these fields.

This PR doese not rename any Golang variables. This should be done in a follow up and in sync with potential changes to the registry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants