Skip to content

[Backport main] Update 3.4 release Note (#4923) #4934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LantaoJin merged 1 commit into from
Dec 11, 2025
Merged

[Backport main] Update 3.4 release Note (#4923) #4934

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 155 additions & 0 deletions release-notes/opensearch-sql.release-notes-3.4.0.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@

## Version 3.4.0 Release Notes

Compatible with OpenSearch and OpenSearch Dashboards version 3.4.0

### Features
* Support `chart` command in PPL ([#4579](https://github.com/opensearch-project/sql/pull/4579))
* Support `Streamstats` command with calcite ([#4297](https://github.com/opensearch-project/sql/pull/4297))
* Support `multisearch` command in calcite ([#4332](https://github.com/opensearch-project/sql/pull/4332))
* Add replace command with Calcite ([#4451](https://github.com/opensearch-project/sql/pull/4451))

### Enhancements
* Add `bucket_nullable` argument for `Streamstats` command ([#4831](https://github.com/opensearch-project/sql/pull/4831))
* Add `regexp_replace()` function as alias of `replace()` ([#4765](https://github.com/opensearch-project/sql/pull/4765))
* Convert `dedup` pushdown to composite + top_hits ([#4844](https://github.com/opensearch-project/sql/pull/4844))
* Merge group fields for aggregate if having dependent group fields ([#4703](https://github.com/opensearch-project/sql/pull/4703))
* Merge the implementation of `timechart` and `chart` ([#4755](https://github.com/opensearch-project/sql/pull/4755))
* PPL tostring() implementation issue #4492 ([#4497](https://github.com/opensearch-project/sql/pull/4497))
* Perform RexNode expression standardization for script push down ([#4795](https://github.com/opensearch-project/sql/pull/4795))
* Pushdown sort by complex expressions to scan ([#4750](https://github.com/opensearch-project/sql/pull/4750))
* Pushdown the `top` `rare` commands to nested aggregation ([#4707](https://github.com/opensearch-project/sql/pull/4707))
* Refactor alias type field by adding another project with alias ([#4881](https://github.com/opensearch-project/sql/pull/4881))
* Remove count aggregation for sort on aggregate measure ([#4867](https://github.com/opensearch-project/sql/pull/4867))
* Remove redundant push-down-filters derived for bucket-non-null agg ([#4843](https://github.com/opensearch-project/sql/pull/4843))
* Remove unnecessary filter for DateHistogram aggregation ([#4877](https://github.com/opensearch-project/sql/pull/4877))
* Specify timestamp field with `timefield` in timechart command ([#4784](https://github.com/opensearch-project/sql/pull/4784))
* Support `appendpipe` command in PPL ([#4602](https://github.com/opensearch-project/sql/pull/4602))
* Support `mvdedup` eval function ([#4828](https://github.com/opensearch-project/sql/pull/4828))
* Support `mvindex` eval function ([#4794](https://github.com/opensearch-project/sql/pull/4794))
* Support push down sort on aggregation measure for more than one agg call ([#4759](https://github.com/opensearch-project/sql/pull/4759))
* Support wildcard for replace command ([#4698](https://github.com/opensearch-project/sql/pull/4698))
* Add `bucket_nullable` argument for `Eventstats` ([#4817](https://github.com/opensearch-project/sql/pull/4817))
* Bin command error message enhancement ([#4690](https://github.com/opensearch-project/sql/pull/4690))
* Update clickbench queries with parameter bucket_nullable=false ([#4732](https://github.com/opensearch-project/sql/pull/4732))
* Support 'usenull' option in PPL `top` and `rare` commands ([#4696](https://github.com/opensearch-project/sql/pull/4696))
* Support millisecond span ([#4672](https://github.com/opensearch-project/sql/pull/4672))
* Enhance dynamic source clause to support only metadata filters ([#4554](https://github.com/opensearch-project/sql/pull/4554))
* Support push down sort after limit ([#4657](https://github.com/opensearch-project/sql/pull/4657))
* Add `per_minute`, `per_hour`, `per_day` function support ([#4531](https://github.com/opensearch-project/sql/pull/4531))
* Pushdown sort aggregate metrics ([#4603](https://github.com/opensearch-project/sql/pull/4603))
* Allow renaming group-by fields to existing field names ([#4586](https://github.com/opensearch-project/sql/pull/4586))
* Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions ([#4599](https://github.com/opensearch-project/sql/pull/4599))
* Pushdown case function in aggregations as range queries ([#4400](https://github.com/opensearch-project/sql/pull/4400))
* Update GEOIP function to support IP types as input ([#4613](https://github.com/opensearch-project/sql/pull/4613))
* Pushdown distinct count approx ([#4614](https://github.com/opensearch-project/sql/pull/4614))
* Optimize pushdown script size with necessary fields per expression ([#4615](https://github.com/opensearch-project/sql/pull/4615))
* Support referring to implicit `@timestamp` field in span ([#4138](https://github.com/opensearch-project/sql/pull/4138))
* Make composite bucket size configurable ([#4544](https://github.com/opensearch-project/sql/pull/4544))
* Add internal MAP_REMOVE function for Calcite PPL ([#4511](https://github.com/opensearch-project/sql/pull/4511))
* Add MAP_APPEND internal function to Calcite PPL ([#4515](https://github.com/opensearch-project/sql/pull/4515))
* Use `_doc` + `_shard_doc` as sort tiebreaker to get better performance ([#4569](https://github.com/opensearch-project/sql/pull/4569))
* [Enhancement] Error handling for illegal character usage in java regex named capture group ([#4434](https://github.com/opensearch-project/sql/pull/4434))
* Add JSON_EXTRACT_ALL internal function for Calcite PPL ([#4489](https://github.com/opensearch-project/sql/pull/4489))
* Set 0 and negative value of subsearch.maxout as unlimited ([#4534](https://github.com/opensearch-project/sql/pull/4534))
* Add configurable system limitations for `subsearch` and `join` command ([#4501](https://github.com/opensearch-project/sql/pull/4501))
* Add `per_second` function support for `timechart` command ([#4464](https://github.com/opensearch-project/sql/pull/4464))
* Add MAP_CONCAT internal function ([#4477](https://github.com/opensearch-project/sql/pull/4477))
* Support Regex for replace eval function ([#4456](https://github.com/opensearch-project/sql/pull/4456))
* Add data anonymizer for spath command ([#4479](https://github.com/opensearch-project/sql/pull/4479))
* Support eval returns decimal division result instead of integer ([#4440](https://github.com/opensearch-project/sql/pull/4440))
* Add mvappend function for Calcite PPL ([#4438](https://github.com/opensearch-project/sql/pull/4438))
* PPL `fillnull` command enhancement ([#4421](https://github.com/opensearch-project/sql/pull/4421))
* Support format=yaml in Explain API ([#4446](https://github.com/opensearch-project/sql/pull/4446))
### Bug Fixes
* Add hashCode() and equals() to the value class of ExprJavaType ([#4885](https://github.com/opensearch-project/sql/pull/4885))
* BucketAggretationParser should handle more non-composite bucket types ([#4706](https://github.com/opensearch-project/sql/pull/4706))
* Do not remove nested fields in resolving AllFieldsExcludeMeta ([#4708](https://github.com/opensearch-project/sql/pull/4708))
* Fix binning udf resolution / Add type coercion support for binning UDFs ([#4742](https://github.com/opensearch-project/sql/pull/4742))
* Fix bug that `Streamstats` command incorrectly treats null as a valid group ([#4777](https://github.com/opensearch-project/sql/pull/4777))
* Fix filter push down producing redundant filter queries ([#4744](https://github.com/opensearch-project/sql/pull/4744))
* Fix function identify problem in converting to sql dialect ([#4793](https://github.com/opensearch-project/sql/pull/4793))
* Fix search anonymizer only ([#4783](https://github.com/opensearch-project/sql/pull/4783))
* Fix sub-fields accessing of generated structs ([#4683](https://github.com/opensearch-project/sql/pull/4683))
* Fix wrong parameter and return result logic for LogPatternAggFunction ([#4868](https://github.com/opensearch-project/sql/pull/4868))
* Grouping key field type can only be overwritten when the `ExprCoreType`s are different ([#4850](https://github.com/opensearch-project/sql/pull/4850))
* Fix eval on grouped fields after timechart ([#4758](https://github.com/opensearch-project/sql/pull/4758))
* Support access to nested field of struct after fields command ([#4719](https://github.com/opensearch-project/sql/pull/4719))
* Support escaped field names in SPath parsing ([#4813](https://github.com/opensearch-project/sql/pull/4813))
* Support script pushdown in sort-on-measure pushdown rewriting ([#4749](https://github.com/opensearch-project/sql/pull/4749))
* Support serializing external OpenSearch UDFs at pushdown time ([#4618](https://github.com/opensearch-project/sql/pull/4618))
* Support using decimal as span literals ([#4717](https://github.com/opensearch-project/sql/pull/4717))
* Translate `SAFE_CAST` to `TRY_CAST` in Spark SQL ([#4788](https://github.com/opensearch-project/sql/pull/4788))
* Update syntax: like(string, PATTERN[, case_sensitive]) ([#4837](https://github.com/opensearch-project/sql/pull/4837))
* [BugFix] Fix Memory Exhaustion for Multiple Filtering Operations in PPL ([#4841](https://github.com/opensearch-project/sql/pull/4841))
* Fix CVE-2025-48924 ([#4665](https://github.com/opensearch-project/sql/pull/4665))
* [BugFix] Fix unexpected shift of extraction for `rex` with nested capture groups in named groups ([#4641](https://github.com/opensearch-project/sql/pull/4641))
* Fix asc/desc keyword behavior for sort command ([#4651](https://github.com/opensearch-project/sql/pull/4651))
* Fixes for `Multisearch` and `Append` command ([#4512](https://github.com/opensearch-project/sql/pull/4512))
* Make nested alias type support referring to outer context ([#4673](https://github.com/opensearch-project/sql/pull/4673))
* Use table scan rowType in filter pushdown to fix rename issue ([#4670](https://github.com/opensearch-project/sql/pull/4670))
* Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations ([#4621](https://github.com/opensearch-project/sql/pull/4621))
* Fix bin nested fields issue ([#4606](https://github.com/opensearch-project/sql/pull/4606))
* Change ComparableLinkedHashMap to compare Key than Value ([#4648](https://github.com/opensearch-project/sql/pull/4648))
* Replace all dots in fields of table scan's PhysType ([#4633](https://github.com/opensearch-project/sql/pull/4633))
* Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue ([#4629](https://github.com/opensearch-project/sql/pull/4629))
* Fix filter parsing failure on date fields with non-default format ([#4616](https://github.com/opensearch-project/sql/pull/4616))
* Fix compile issue in main ([#4608](https://github.com/opensearch-project/sql/pull/4608))
* Fix push down failure for min/max on derived field ([#4572](https://github.com/opensearch-project/sql/pull/4572))
* Add value type hint for derived aggregate group by field ([#4583](https://github.com/opensearch-project/sql/pull/4583))
* Fix sort push down into agg after project already pushed ([#4546](https://github.com/opensearch-project/sql/pull/4546))
* Update request builder after pushdown sort into agg buckets ([#4541](https://github.com/opensearch-project/sql/pull/4541))
* Including metadata fields type when doing agg/filter script push down ([#4522](https://github.com/opensearch-project/sql/pull/4522))
* Fix percentile bug ([#4539](https://github.com/opensearch-project/sql/pull/4539))
* Fix mapping after aggregation push down ([#4500](https://github.com/opensearch-project/sql/pull/4500))
* Throw an error when the conditions of case are not boolean values ([#4520](https://github.com/opensearch-project/sql/pull/4520))
* Fallback to sub-aggregation if composite aggregation doesn't support ([#4413](https://github.com/opensearch-project/sql/pull/4413))
* Fix the bug of explicit makeNullLiteral for UDT fields ([#4475](https://github.com/opensearch-project/sql/pull/4475))
* Fix missing keywordsCanBeId ([#4491](https://github.com/opensearch-project/sql/pull/4491))
* Fix issue 4441 ([#4449](https://github.com/opensearch-project/sql/pull/4449))
* Fix join type ambiguous issue when specify the join type with sql-like join criteria ([#4474](https://github.com/opensearch-project/sql/pull/4474))
* Remove shared mutable optimizer field that caused race condition ([#4454](https://github.com/opensearch-project/sql/pull/4454))
* Reverting to _doc + _id ([#4435](https://github.com/opensearch-project/sql/pull/4435))

### Infrastructure
* Add config for CodeRabbit review ([#4890](https://github.com/opensearch-project/sql/pull/4890))
* Split bwc-tests to bwc-rolling-upgrade and bwc-full-restart ([#4716](https://github.com/opensearch-project/sql/pull/4716))
* Update github workflows to move from macos-13 to 14 ([#4779](https://github.com/opensearch-project/sql/pull/4779))
* Fix the flaky CalcitePPLTcphIT ([#4846](https://github.com/opensearch-project/sql/pull/4846))
* Fix UT failure and Linkchecker failure ([#4809](https://github.com/opensearch-project/sql/pull/4809))
* Adding IT suite for PPL-based dashboards in Neo for CloudWatch Lake ([#4695](https://github.com/opensearch-project/sql/pull/4695))
* Add allowed_warnings in yaml restful tests ([#4731](https://github.com/opensearch-project/sql/pull/4731))
* Mitigate the CI failure caused by 500 Internal Server Error ([#4646](https://github.com/opensearch-project/sql/pull/4646))
* Publish internal modules separately for downstream reuse ([#4484](https://github.com/opensearch-project/sql/pull/4484))
* Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT ([#4623](https://github.com/opensearch-project/sql/pull/4623))
* Onboarding async query core and grammar files to maven snapshots ([#4598](https://github.com/opensearch-project/sql/pull/4598))
* Onboarding new maven snapshots publishing to s3 (sql) ([#4588](https://github.com/opensearch-project/sql/pull/4588))
* Fix JsonExtractAllFunctionIT failure ([#4556](https://github.com/opensearch-project/sql/pull/4556))
* Check server status before starting Prometheus ([#4537](https://github.com/opensearch-project/sql/pull/4537))
* Update stalled action ([#4485](https://github.com/opensearch-project/sql/pull/4485))
* Switch to Guice#createInjector and add concurrent SQL/PPL regression ITs ([#4462](https://github.com/opensearch-project/sql/pull/4462))
* Update delete_backport_branch workflow to include release-chores branches ([#4025](https://github.com/opensearch-project/sql/pull/4025))
* Add ignorePrometheus Flag for integTest and docTest ([#4442](https://github.com/opensearch-project/sql/pull/4442))

### Documentation
* Update PPL Command Documentation ([#4562](https://github.com/opensearch-project/sql/pull/4562))
* Doc update for `json_valid` ([#4803](https://github.com/opensearch-project/sql/pull/4803))
* Enhance tests and doc for eval isnull/isnotnull functions ([#4724](https://github.com/opensearch-project/sql/pull/4724))
* Update search.rst documentation ([#4686](https://github.com/opensearch-project/sql/pull/4686))
* Add more examples to the `where` command doc ([#4457](https://github.com/opensearch-project/sql/pull/4457))
* Update eventstats.rst ([#4447](https://github.com/opensearch-project/sql/pull/4447))

### Maintenance
* Bump Calcite to 1.41.0 ([#4714](https://github.com/opensearch-project/sql/pull/4714))
* Execute yamlRestTest in integration job ([#4838](https://github.com/opensearch-project/sql/pull/4838))
* Fix test failures due to version in mapping ([#4748](https://github.com/opensearch-project/sql/pull/4748))
* Support timeouts for Calcite queries ([#4857](https://github.com/opensearch-project/sql/pull/4857))
* [Maintenance] Enforce PR label of 'bugFix' instead of 'bug' ([#4773](https://github.com/opensearch-project/sql/pull/4773))
* [3.4.0] Bump Gradle to 9.2.0 and GitHub Action JDK to 25 ([#4824](https://github.com/opensearch-project/sql/pull/4824))
* Fix clickbench query 43 ([#4861](https://github.com/opensearch-project/sql/pull/4861))
* Update big5 ppl queries and check plans ([#4668](https://github.com/opensearch-project/sql/pull/4668))
* Revert "Update grammar files and developer guide (#4301)" ([#4643](https://github.com/opensearch-project/sql/pull/4643))
* Increment version to 3.4.0-SNAPSHOT ([#4452](https://github.com/opensearch-project/sql/pull/4452))
* Revert partial of #4401 ([#4503](https://github.com/opensearch-project/sql/pull/4503))
* Implement one-batch lookahead for index enumerators ([#4345](https://github.com/opensearch-project/sql/pull/4345))
* Refactor name resolution in Calcite PPL ([#4393](https://github.com/opensearch-project/sql/pull/4393))
Loading