Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TSDB: Make time boundaries settings required in tsdb indices #81146

Merged
merged 12 commits into from
Dec 1, 2021
Merged

TSDB: Make time boundaries settings required in tsdb indices #81146

merged 12 commits into from
Dec 1, 2021

Conversation

weizijun
Copy link
Contributor

Make index.time_series.start_time and index.time_series.end_time settings as required settings in tsdb indices.

It will change many tests, among which a time_series index is created, and it will add the index.time_series.start_time and index.time_series.end_time settings

@weizijun
temp
c9bfdc4
@weizijun
Merge branch 'upstream/master' into tsdb-required-range-settings
21b60b5 
* upstream/master: (29 commits)
  Fix typo (elastic#80925)
  Increase docker compose timeouts for CI builds
  TSDB: fix error without feature flag (elastic#80945)
  [DOCS] Relocate `index.mapping.dimension_fields.limit` setting docs (elastic#80964)
  Explicit write methods for always-missing values (elastic#80958)
  TSDB: move TimeSeriesModeIT to yaml tests (elastic#80933)
  [ML] Removing temporary debug (elastic#80956)
  Remove unused ConnectTransportException#node (elastic#80944)
  Reinterpret dots in field names as object structure (elastic#79922)
  Remove obsolete typed legacy index templates (elastic#80937)
  Remove unnecessary shuffle in unassigned shards allocation. (elastic#65172)
  TSDB: Tests for nanosecond timeprecision timestamp just beyond the limit (elastic#80932)
  Cleanup SLM History Item .equals (elastic#80938)
  Rework breaking changes for new structure (elastic#80907)
  [DOCS] Fix elasticsearch-reset-password typo (elastic#80919)
  [ML] No need to use parent task client when internal infer delegates (elastic#80905)
  Fix shadowed vars pt6 (elastic#80899)
  add ignore info (elastic#80924)
  Fix several potential circuit breaker leaks in Aggregators (elastic#79676)
  Extract more standard metadata from binary files (elastic#78754)
  ...
@weizijun
fixup
bc139db
@weizijun
fixup
19473d7
@weizijun
fixup
d025a4e
@weizijun
Merge branch 'upstream/master' into tsdb-required-range-settings
7d3789c 
* upstream/master: (55 commits)
  Fix ComposableIndexTemplate equals when composed_of is null (elastic#80864)
  Optimize DLS bitset building for matchAll query (elastic#81030)
  URL option for BaseRunAsSuperuserCommand (elastic#81025)
  Less Verbose Serialization of Snapshot Failure in SLM Metadata (elastic#80942)
  Fix shadowed vars pt7 (elastic#80996)
  Fail shards early when we can detect a type missmatch (elastic#79869)
  Delegate Ref Counting to ByteBuf in Netty Transport (elastic#81096)
  Clarify `unassigned.reason` docs (elastic#81017)
  Strip blocks from settings for reindex targets (elastic#80887)
  Split off the values supplier for ScriptDocValues (elastic#80635)
  [ML] Switch message and detail for model snapshot deprecations (elastic#81108)
  [DOCS] Update xrefs for snapshot restore docs (elastic#81023)
  [ML] Updates visiblity of validate API (elastic#81061)
  Track histogram of transport handling times (elastic#80581)
  [ML] Fix datafeed preview with remote indices (elastic#81099)
  [ML] Fix acceptable model snapshot versions in ML deprecation checker (elastic#81060)
  [ML] Add logging for failing PyTorch test (elastic#81044)
  Extending the timeout waiting for snapshot to be ready (elastic#81018)
  [ML] Fix incorrect logging of unexpected model size error (elastic#81089)
  [ML] Make inference timeout test more reliable (elastic#81094)
  ...
@weizijun
update tests
e6765aa
@elasticsearchmachine elasticsearchmachine added v8.1.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Nov 30, 2021
@pgomulka pgomulka added the :StorageEngine/TSDB You know, for Metrics label Nov 30, 2021
@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 30, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

nik9000
nik9000 reviewed Nov 30, 2021
View reviewed changes
Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be nice to use times in the tests that are a little closer to the test data. If we're really make truly random times from 1 to Long.MAX_VALUE it's cool to have those as bounds, but for the tests I'd feel better with bounds that are a day or a month long or something like that.

rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/tsdb/15_timestamp_mapping.yml Outdated
@@ -13,6 +13,9 @@ date:
index:
mode: time_series
routing_path: [metricset]
time_series:
start_time: 1
end_time: 253402300799998
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be nice to have something a little more realistic here - probably jut 2021-04-28T00:00:00Z to 2021-04-29 or something like that.

@nik9000
Copy link
Member

nik9000 commented Nov 30, 2021

@elasticmachine, test this please

@nik9000 nik9000 added the test-release Trigger CI checks against release build label Nov 30, 2021
@weizijun
update start_time and end_time in tests
cbf1ad0
@weizijun
Merge branch 'upstream/master' into tsdb-required-range-settings
87c8713 
* upstream/master:
  [ML] Parent datafeed actions to the datafeed's persistent task (elastic#81143)
  Simplify ConcreteIndices and its usage in TransportBulkAction (elastic#81098)
  Unmute DataStreamsSnapshotsIT#testRestoreDataStreamAliasWithConflictingIndicesAlias() test (elastic#81142)
  TSDB: Do not allow index splits for time series indices (elastic#81125)
  Reduce verbosity-increase timeout to 3m (elastic#81118)
  Mute DataStreamsSnapshotsIT#testRestoreDataStreamAliasWithConflictingIndicesAlias() test
  Fix stopping of old elasticsearch cluster (elastic#81059)
  Fix data stream alias validation. (elastic#81040)
  Add replicated field to get data stream api response. (elastic#80988)
@weizijun
Copy link
Contributor Author

@nik9000 can I change all tsdb yaml test's skip config from:

      version: " - 7.99.99"
      reason: introduced in 8.0.0

to

      version: " - 8.0.99"
      reason: introduced in 8.1.0

to pass the mixed-cluster tests

@nik9000
Copy link
Member

nik9000 commented Nov 30, 2021

to pass the mixed-cluster tests

👍

@nik9000
Copy link
Member

nik9000 commented Nov 30, 2021

@elasticmachine, test this please

@weizijun
Copy link
Contributor Author

weizijun commented Dec 1, 2021

@elasticmachine update branch

@weizijun weizijun changed the title Make time boundaries settings required in tsdb indices TSDB: Make time boundaries settings required in tsdb indices Dec 1, 2021
@weizijun
Copy link
Contributor Author

weizijun commented Dec 1, 2021

@nik9000 I have passed all check. can you help to merge the PR?

@nik9000
Copy link
Member

nik9000 commented Dec 1, 2021

@elasticmachine, test this please

@nik9000 nik9000 mentioned this pull request Dec 1, 2021
Closed
@weizijun
Copy link
Contributor Author

weizijun commented Dec 1, 2021

@nik9000 the failed elasticsearch-ci/release-tests seems that the problem of the environment

@nik9000
Copy link
Member

nik9000 commented Dec 1, 2021

run elasticsearch-ci/release-tests

@nik9000
Copy link
Member

nik9000 commented Dec 1, 2021

@nik9000 the failed elasticsearch-ci/release-tests seems that the problem of the environment

👍. Let's see if it's better a second time.

@nik9000
Copy link
Member

nik9000 commented Dec 1, 2021

run elasticsearch-ci/release-tests

@nik9000
Copy link
Member

nik9000 commented Dec 1, 2021

@elasticmachine, update branch

@nik9000 nik9000 merged commit 2f855aa into elastic:master Dec 1, 2021
@dimitris-athanasiou dimitris-athanasiou mentioned this pull request Dec 13, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nik9000 nik9000 nik9000 left review comments

Assignees
No one assigned
Labels
>enhancement external-contributor Pull request authored by a developer outside the Elasticsearch team :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) test-release Trigger CI checks against release build v8.1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@weizijun @elasticmachine @nik9000 @pugnascotia @pgomulka @elasticsearchmachine