Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Manager] Adjust dataset aggs to use datastream fields instead #74342

Merged
merged 6 commits into from
Aug 5, 2020
Merged

[Ingest Manager] Adjust dataset aggs to use datastream fields instead #74342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7589eff
[Ingest Manager] Adjust dataset aggs to use datastream fields instead
ruflin Aug 4, 2020
89de0ec
Update datastream to data_stream
jen-huang Aug 4, 2020
794d3c6
Update data stream name generation
jen-huang Aug 4, 2020
0b73367
Fix typo
jen-huang Aug 4, 2020
e3e42a3
Temporarily use datastream instead of data_stream
jen-huang Aug 4, 2020
76b94b8
updating to use `data_stream` instead of `datastream`
ruflin Aug 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions x-pack/plugins/ingest_manager/server/routes/data_streams/handlers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ export const getListHandler: RequestHandler = async (context, request, response)
must: [
{
exists: {
field: 'dataset.namespace',
field: 'data_stream.namespace',
},
},
{
exists: {
field: 'dataset.name',
field: 'data_stream.dataset',
},
},
],
Expand All @@ -54,19 +54,19 @@ export const getListHandler: RequestHandler = async (context, request, response)
aggs: {
dataset: {
terms: {
field: 'dataset.name',
field: 'data_stream.dataset',
size: 1,
},
},
namespace: {
terms: {
field: 'dataset.namespace',
field: 'data_stream.namespace',
size: 1,
},
},
type: {
terms: {
field: 'dataset.type',
field: 'data_stream.type',
size: 1,
},
},
Expand Down
12 changes: 6 additions & 6 deletions x-pack/plugins/ingest_manager/server/services/epm/elasticsearch/template/template.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,14 +393,14 @@ const updateExistingIndex = async ({
// are added in https://github.com/elastic/kibana/issues/66551. namespace value we will continue
// to skip updating and assume the value in the index mapping is correct
delete mappings.properties.stream;
delete mappings.properties.dataset;
delete mappings.properties.data_stream;

// get the dataset values from the index template to compose data stream name
// get the data_stream values from the index template to compose data stream name
const indexMappings = await getIndexMappings(indexName, callCluster);
const dataset = indexMappings[indexName].mappings.properties.dataset.properties;
if (!dataset.type.value || !dataset.name.value || !dataset.namespace.value)
throw new Error(`dataset values are missing from the index template ${indexName}`);
const dataStreamName = `${dataset.type.value}-${dataset.name.value}-${dataset.namespace.value}`;
const dataStream = indexMappings[indexName].mappings.properties.data_stream.properties;
if (!dataStream.type.value || !dataStream.dataset.value || !dataStream.namespace.value)
throw new Error(`data_stream values are missing from the index template ${indexName}`);
const dataStreamName = `${dataStream.type.value}-${dataStream.dataset.value}-${dataStream.namespace.value}`;

// try to update the mappings first
try {
Expand Down