Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Star tree] Adding date field rounding support in star tree #15249

Merged
merged 16 commits into from
Oct 8, 2024

Conversation

bharath-techie
Copy link
Contributor

@bharath-techie bharath-techie commented Aug 14, 2024

Description

User can give up to three calendar_intervals as part of Date_Dimension as part of star tree mapping. We will round off the value in date field as per the interval. Both nano and millis resolution are rounded off.

Anything greater than or equal to Hour interval will only work for UTC timezone and 00:00 offset.

We will add support for timezones and offsets later. #15266

        "composite": {
            "startree1: {
                "type": "star_tree",
                "config": {
                    "ordered_dimensions": [
                        {
                            "name": "size"
                        }
                    ],
                    "date_dimension": {
                        "name": "date",
                        "calendar_intervals": [
                            "minute",
                            "hour",
                             "day"
                        ]
                    },
                    "metrics": [
                        {
                            "name": "size",
                            "stats": [
                                "sum",
                                "count"
                            ]
                        }
                    ]
                }
            }
        }

User can specify date dimension as above with defaults being 'minute' and 'half an hour'. The date field ( both milliseconds and nanoseconds resolution ) will get rounded off to the associated calendar interval.

Apart from date histogram calendar intervals,
User can provide quarter of hour / half an hour as part of mapping .

15m (or) quarter-hour
30m (or) half-hour

This is done to handle all timezones.
15m interval will handle all timezones and 30m interval will handle 99% of timezones bar the ones which has :45 minute part as part of timezone.

Related Issues

Resolves #15265

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 5fdce71: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 23e600a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 6fbaf1b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
@github-actions github-actions bot added the enhancement Enhancement or improvement to existing feature or request label Aug 15, 2024
Copy link
Contributor

❌ Gradle check result for 93760bc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Copy link
Contributor

github-actions bot commented Oct 1, 2024

❕ Gradle check result for d34a36f: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Copy link
Contributor

github-actions bot commented Oct 4, 2024

❌ Gradle check result for 309f539: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 4, 2024

❌ Gradle check result for d9e7852: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 4, 2024

✅ Gradle check result for 8215865: SUCCESS

@sachinpkale sachinpkale merged commit 1e49aa8 into opensearch-project:main Oct 8, 2024
33 of 34 checks passed
@bharath-techie bharath-techie added the backport 2.x Backport to 2.x branch label Oct 14, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 14, 2024
---------

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
(cherry picked from commit 1e49aa8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
linuxpi pushed a commit that referenced this pull request Oct 15, 2024
…6304)

---------


(cherry picked from commit 1e49aa8)

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 16, 2024
…project#15249)

---------

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 17, 2024
…project#15249)

---------

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 21, 2024
…project#15249)

---------

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch enhancement Enhancement or improvement to existing feature or request Indexing:Performance skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Star tree] Rounding off date field in Star tree
5 participants