Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Indexing Strategy] Change stream.* to dataset.* fields #491

Closed
8 tasks done
ruflin opened this issue Jun 3, 2020 · 1 comment · Fixed by elastic/beats#18967
Closed
8 tasks done

[Indexing Strategy] Change stream.* to dataset.* fields #491

ruflin opened this issue Jun 3, 2020 · 1 comment · Fixed by elastic/beats#18967
Assignees
@ruflin
Labels
Ingest Management:beta1 Group issues for ingest management beta1

Comments

@ruflin
Copy link
Member

ruflin commented Jun 3, 2020 •
edited
Loading

In #482 it was discussed and decided to move from stream.* fields to dataset.* fields. This issue is to track the changes across all the different affected parts across the stack.

It might be that some of the above changes will be breaking and a sync is needed between teams to get them merged at the same time.
@ruflin ruflin added the Ingest Management:beta1 Group issues for ingest management beta1 label Jun 3, 2020
@ruflin ruflin self-assigned this Jun 3, 2020
@ruflin ruflin mentioned this issue Jun 3, 2020
Closed
ruflin added a commit to ruflin/package-registry that referenced this issue Jun 3, 2020
@ruflin
Update registry for stream.* to dataset.* change
d380ee1 
Now the dataset intsead of the stream fields are added and validated.

Part of elastic#491
ruflin added a commit to ruflin/package-registry that referenced this issue Jun 3, 2020
@ruflin
Update registry for stream.* to dataset.* change
4f5b7ff 
Now the dataset intsead of the stream fields are added and validated.

Updating also the base package. This will have to be pushed to the package-storage.

Part of elastic#491
ruflin added a commit to ruflin/package-registry that referenced this issue Jun 3, 2020
@ruflin
Update registry for stream.* to dataset.* change
ef63582 
Now the dataset intsead of the stream fields are added and validated.

Updating also the base package. This will have to be pushed to the package-storage.

Part of elastic#491
This was referenced Jun 3, 2020
Merged
Merged
@ph ph mentioned this issue Jun 3, 2020
Closed
@ph ph mentioned this issue Jun 3, 2020
Closed
ruflin added a commit to elastic/kibana that referenced this issue Jun 3, 2020
@ruflin
[Ingest Manager] Update indexing strategy docs to use dataset.* (#68068)
055880d 
Indexing strategy now uses dataset.* instead of stream.* fields. For this the indexing strategy docs are updated.

Part of elastic/package-registry#491
@ruflin ruflin mentioned this issue Jun 4, 2020
Merged
@michalpristas michalpristas mentioned this issue Jun 4, 2020
Merged
6 tasks
@jen-huang jen-huang mentioned this issue Jun 4, 2020
Merged
ruflin added a commit to ruflin/package-registry that referenced this issue Jun 8, 2020
@ruflin
Update registry for stream.* to dataset.* change
0207a99 
Now the dataset intsead of the stream fields are added and validated.

Updating also the base package. This will have to be pushed to the package-storage.

Part of elastic#491
ruflin added a commit to ruflin/package-storage that referenced this issue Jun 8, 2020
@ruflin
Add dataset fields to all packages
f1f503e 
This is part of elastic/package-registry#491 This PR adds the new dataset fields to each package. As soon as all PRs in the registry, Kibana and Agent are merged, the old stream fields can be removed.
@ruflin ruflin mentioned this issue Jun 8, 2020
Merged
ruflin added a commit to elastic/package-storage that referenced this issue Jun 8, 2020
@ruflin
Add dataset fields to all packages (#56)
53dfc42 
This is part of elastic/package-registry#491 This PR adds the new dataset fields to each package. As soon as all PRs in the registry, Kibana and Agent are merged, the old stream fields can be removed.
ruflin added a commit that referenced this issue Jun 8, 2020
@ruflin
Update registry for stream.* to dataset.* change (#492)
955239d 
Now the dataset intsead of the stream fields are added and validated.

Updating also the base package. This will have to be pushed to the package-storage.

Part of #491
ruflin pushed a commit to elastic/kibana that referenced this issue Jun 9, 2020
@jen-huang
[Ingest Manager] Update queries from stream.* to dataset.* (#68322)
bc6aadd 
Resolves #67672

Part of elastic/package-registry#491

See elastic/package-registry#492 for package registry changes.

Updates existing queries to use `dataset.*` fields. from `stream.*` fields.

I looked at the EPM code paths for asset installation, and did not find anything that would need to be changed based on the above package registry changes. We don't hardcode anything specific for field names. The package information (i.e. [nginx package info](https://epr.elastic.co/package/nginx/0.1.2/)) is not affected by the current package registry changes.
@ruflin ruflin reopened this Jun 9, 2020
jen-huang added a commit to jen-huang/kibana that referenced this issue Jun 9, 2020
@jen-huang
[Ingest Manager] Update queries from stream.* to dataset.* (elast…
9c89854 
…ic#68322)

Resolves elastic#67672

Part of elastic/package-registry#491

See elastic/package-registry#492 for package registry changes.

Updates existing queries to use `dataset.*` fields. from `stream.*` fields.

I looked at the EPM code paths for asset installation, and did not find anything that would need to be changed based on the above package registry changes. We don't hardcode anything specific for field names. The package information (i.e. [nginx package info](https://epr.elastic.co/package/nginx/0.1.2/)) is not affected by the current package registry changes.
jen-huang added a commit to elastic/kibana that referenced this issue Jun 9, 2020
@jen-huang
[Ingest Manager] Update queries from stream.* to dataset.* (#68322)…
73d30d9 
… (#68667)

Resolves #67672

Part of elastic/package-registry#491

See elastic/package-registry#492 for package registry changes.

Updates existing queries to use `dataset.*` fields. from `stream.*` fields.

I looked at the EPM code paths for asset installation, and did not find anything that would need to be changed based on the above package registry changes. We don't hardcode anything specific for field names. The package information (i.e. [nginx package info](https://epr.elastic.co/package/nginx/0.1.2/)) is not affected by the current package registry changes.
@michalpristas michalpristas mentioned this issue Jun 10, 2020
Merged
6 tasks
@ruflin
Copy link
Member Author

ruflin commented Aug 18, 2020

Closing, as this was completed and we even followed up renaming things to data_stream.* ...

@ruflin ruflin closed this as completed Aug 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ruflin ruflin

Labels
Ingest Management:beta1 Group issues for ingest management beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Ingest Manager] Moved from stream to dataset michalpristas/beats
2 participants
@ph @ruflin