Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set default index mode for TimeSeries to null #98808

Merged
merged 26 commits into from
Aug 25, 2023
Merged

Conversation

kkrik-es
Copy link
Contributor

If the default index mode matches the specified, we skip printing the synthetic source info in mappings. This leads to confusion as it's not immediately visible (or well known) that time series indices use synthetic source by default.

Leaving the default index mode to null does the trick here. We do pass the right value for time series indexes while building the mapping so there's no other functional impact.

This was attempted in #98586, got rolled back because it broke bwc #98803. Rolling fwd with a fix for this.

Fixes #97429

kkrik-es and others added 12 commits August 8, 2023 19:11
When a query of a filter gets rewritten to MatchNoDocsQuery, segments
will not produce any results. We can therefore skip processing such
segments.

This applies to FilterByFilterAggregator and FiltersAggregator, as well
as to TermsAggregator when it uses StringTermsAggregatorFromFilters
internally; the latter is an adapter aggregator to
FilterByFilterAggregator.

Fixes elastic#94637
@kkrik-es kkrik-es added >enhancement :Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) auto-backport Automatically create backport pull requests when merged v8.10.1 labels Aug 23, 2023
@kkrik-es kkrik-es self-assigned this Aug 23, 2023
@elasticsearchmachine
Copy link
Collaborator

Hi @kkrik-es, I've created a changelog YAML for you.

@kkrik-es kkrik-es added the test-full-bwc Trigger full BWC version matrix tests label Aug 23, 2023
@kkrik-es kkrik-es requested a review from martijnvg August 24, 2023 10:33
@kkrik-es kkrik-es marked this pull request as ready for review August 24, 2023 10:33
@kkrik-es
Copy link
Contributor Author

@elasticsearchmachine run elasticsearch-ci/full-bwc

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot to say this in the previous PR, but maybe we should add a rest test (that checks that source mode is included in template simulate api)?
Otherwise LGTM.

@kkrik-es
Copy link
Contributor Author

I forgot to say this in the previous PR, but maybe we should add a rest test (that checks that source mode is included in template simulate api)? Otherwise LGTM.

Good call, added the test.

@kkrik-es kkrik-es merged commit 524ecfb into elastic:main Aug 25, 2023
kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Aug 25, 2023
* Skip segment for MatchNoDocsQuery filters.

When a query of a filter gets rewritten to MatchNoDocsQuery, segments
will not produce any results. We can therefore skip processing such
segments.

This applies to FilterByFilterAggregator and FiltersAggregator, as well
as to TermsAggregator when it uses StringTermsAggregatorFromFilters
internally; the latter is an adapter aggregator to
FilterByFilterAggregator.

Fixes elastic#94637

* Update docs/changelog/98295.yaml

* Check all filters for `MatchNoDocsQuery`.

* Skip optimization when 'other' bucket is requested.

* Revert "Set default index mode for TimeSeries to `null` (elastic#98586)"

This reverts commit 56abb86.

* Revert "Rollback of elastic#98586 (elastic#98805)"

This reverts commit e370194.

* Skip updating source when missing synthetic mode

* Update docs/changelog/98808.yaml

* Skip matching assert in MapperService too

* Refine the assert

* Extend versions before 8.6, when TS had no synthetic source

* Add source field mapping for non-synthetic TSDB

* Delete 98586.yaml

Duplicate changelog

* Add comment to TSDB_NO_SYNTHETIC mapping

* Spotless fix

* Add yaml test

* Fix version skip in yaml test
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.10

elasticsearchmachine pushed a commit that referenced this pull request Aug 25, 2023
* Skip segment for MatchNoDocsQuery filters.

When a query of a filter gets rewritten to MatchNoDocsQuery, segments
will not produce any results. We can therefore skip processing such
segments.

This applies to FilterByFilterAggregator and FiltersAggregator, as well
as to TermsAggregator when it uses StringTermsAggregatorFromFilters
internally; the latter is an adapter aggregator to
FilterByFilterAggregator.

Fixes #94637

* Update docs/changelog/98295.yaml

* Check all filters for `MatchNoDocsQuery`.

* Skip optimization when 'other' bucket is requested.

* Revert "Set default index mode for TimeSeries to `null` (#98586)"

This reverts commit 56abb86.

* Revert "Rollback of #98586 (#98805)"

This reverts commit e370194.

* Skip updating source when missing synthetic mode

* Update docs/changelog/98808.yaml

* Skip matching assert in MapperService too

* Refine the assert

* Extend versions before 8.6, when TS had no synthetic source

* Add source field mapping for non-synthetic TSDB

* Delete 98586.yaml

Duplicate changelog

* Add comment to TSDB_NO_SYNTHETIC mapping

* Spotless fix

* Add yaml test

* Fix version skip in yaml test
bpintea pushed a commit to bpintea/elasticsearch that referenced this pull request Aug 25, 2023
* Skip segment for MatchNoDocsQuery filters.

When a query of a filter gets rewritten to MatchNoDocsQuery, segments
will not produce any results. We can therefore skip processing such
segments.

This applies to FilterByFilterAggregator and FiltersAggregator, as well
as to TermsAggregator when it uses StringTermsAggregatorFromFilters
internally; the latter is an adapter aggregator to
FilterByFilterAggregator.

Fixes elastic#94637

* Update docs/changelog/98295.yaml

* Check all filters for `MatchNoDocsQuery`.

* Skip optimization when 'other' bucket is requested.

* Revert "Set default index mode for TimeSeries to `null` (elastic#98586)"

This reverts commit 56abb86.

* Revert "Rollback of elastic#98586 (elastic#98805)"

This reverts commit e370194.

* Skip updating source when missing synthetic mode

* Update docs/changelog/98808.yaml

* Skip matching assert in MapperService too

* Refine the assert

* Extend versions before 8.6, when TS had no synthetic source

* Add source field mapping for non-synthetic TSDB

* Delete 98586.yaml

Duplicate changelog

* Add comment to TSDB_NO_SYNTHETIC mapping

* Spotless fix

* Add yaml test

* Fix version skip in yaml test
@JVerwolf JVerwolf added v8.10.0 and removed v8.10.1 labels Aug 31, 2023
elasticsearchmachine pushed a commit that referenced this pull request Oct 16, 2023
Yet another test affected by the fix for showing the synthetic source,
#98808. This can trigger an assert in older versions as the mapping they
produce (without synthetic source) doesn't match the one they may get
from the master, if the latter is in version 8.10+.

Fixes #100913
elasticsearchmachine pushed a commit that referenced this pull request Oct 19, 2023
Yet another test affected by the fix for showing the synthetic source,
#98808. This can trigger an assert in older versions as the mapping they
produce (without synthetic source) doesn't match the one they may get
from the master, if the latter is in version 8.10+.

Fixes #101121
kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Oct 19, 2023
Yet another test affected by the fix for showing the synthetic source,
elastic#98808. This can trigger an assert in older versions as the mapping they
produce (without synthetic source) doesn't match the one they may get
from the master, if the latter is in version 8.10+.

Fixes elastic#101121
elasticsearchmachine pushed a commit that referenced this pull request Oct 19, 2023
Yet another test affected by the fix for showing the synthetic source,
#98808. This can trigger an assert in older versions as the mapping they
produce (without synthetic source) doesn't match the one they may get
from the master, if the latter is in version 8.10+.

Fixes #101121
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations auto-backport Automatically create backport pull requests when merged >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) test-full-bwc Trigger full BWC version matrix tests v8.10.0 v8.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Synthetic source is on by default for TSDS but does not appear in mappings
4 participants