fix(sqlsmith): completely cover all exprs #3737
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lmatz
approved these changes
Jul 8, 2022
Codecov Report
@@ Coverage Diff @@
## main #3737 +/- ##
==========================================
- Coverage 74.18% 74.18% -0.01%
==========================================
Files 797 797
Lines 112365 112374 +9
==========================================
+ Hits 83358 83361 +3
- Misses 29007 29013 +6
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more |
huangjw806
approved these changes
Jul 8, 2022
Little-Wallace
added a commit
that referenced
this pull request
Jul 14, 2022
commit 048587d Author: lmatz <lmatz823@gmail.com> Date: Thu Jul 14 11:03:48 2022 +0800 fix(test): pass user name for regress tests (#3854) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 9b0d3a1 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Thu Jul 14 10:13:51 2022 +0800 feat(sqlsmith): generate group by (#3805) * implement sqlsmith group by * fmt * add required cols * pass valid cols to gen_col * gen col from valid_cols if available * override bound columns for group by * capture local column context * fix case where no FROM * fmt * cleanup * clean Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8231cf1 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Thu Jul 14 00:23:16 2022 +0800 feat(row-based encoding): introduce row-based encoding by using value encoding (#3835) * introduce row-based encoding 1.0 commit 1966c93 Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Wed Jul 13 17:08:09 2022 +0200 chore(proto): remove unused FilterScanNode (#3846) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 23388b9 Author: Wenzhuo Liu <lwzbill@foxmail.com> Date: Wed Jul 13 22:48:40 2022 +0800 feat(batch): support selective aggregation (#3683) * feat(batch): support selective aggregation for count * add e2e test for selective count * add group by test for selective count * remove option * add filter to approx count distinct * add general agg and fix approx count * add general distinct agg * format * add e2e test * add selective distinct agg e2e * avoid collect * fix count star * format * fix bug in optimizer Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 2a8a96d Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Wed Jul 13 16:07:03 2022 +0200 style(stream): exhaustive patterm for build_executor (#3848) commit ca49f6d Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 20:55:55 2022 +0800 feat(storage): support deserialize_prefix_len for prefix_bloom_filter (#3807) * feat(storage): support deserialize_prefix_len for prefix_bloom_filter * fix(storage): fix deserialize_prefix unit-test * chore(storage): add doc for encoding_data_size * fix(storage): fix encoding_data_size to return truely length in storage_encoding commit edc6660 Author: Dylan <chenzl25@mail2.sysu.edu.cn> Date: Wed Jul 13 20:38:23 2022 +0800 feat(frontend): support explain verbose (#3798) * feat(frontend) support explain verbose * prefix each field name with table_name * do-apply-planner-test * minor improvement * fix batch two phase agg which should satisfy all agg-call contain no distinct and order by * fix stream two phase agg which should satisfy all agg-call contain no distinct and order by * move explain_verbose flag from session config to optimizer context * fmt * fix test case commit 4a5503e Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 18:59:48 2022 +0800 fix(hash): fix hash_key_data_size (#3843) * fix(hash): fix hash_key_data_size * fix(hash): support Key8 for Boolean commit d0027c6 Author: Bugen Zhao <i@bugenzhao.com> Date: Wed Jul 13 17:53:04 2022 +0800 fix(test): datagen parallel e2e (#3840) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit aa09e44 Author: Huangjw <1223644280@qq.com> Date: Wed Jul 13 17:40:49 2022 +0800 chore(risedev): add external_labels and remote_write in prometeus config (#3825) * add remote write in prometheus config * fix config * add env PROMETHEUS_REMOTE_WRITE * cargo fmt * cargo fmt * update risedev yml * fix risedev yml * fix risedev yml Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6e45426 Author: Alex Chi <iskyzh@gmail.com> Date: Wed Jul 13 17:08:41 2022 +0800 chore(streaming): check epoch continuous (#3839) * chore(streaming): check epoch continuous Signed-off-by: Alex Chi <iskyzh@gmail.com> * fix unit test Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4fadf48 Author: Kaige Li <55606560+likg227@users.noreply.github.com> Date: Wed Jul 13 16:36:10 2022 +0800 feat(optimizer): distinct agg. (#3717) * feat(optimizer): distinct agg. * add comments. * add comments and tests. * correct planner test. * refactor. * improvement. * small fix. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 0f8148d Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Wed Jul 13 16:15:17 2022 +0800 feat(common): support generating `Datum` in field generator (#3827) * add generate_datum to generator traits * add MIN and MAX to OrderedF32 and OrderedF64 * make OrderedFloat serializable and expose impl_rand in common configs * implement generate_datum * add unit tests and fix format * fix bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4b511e8 Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Wed Jul 13 15:36:04 2022 +0800 feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap (#3680) * feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap * add docs Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 369b57e Author: Bohan Zhang <tabvision@bupt.icu> Date: Wed Jul 13 14:42:22 2022 +0800 refactor: move source connectors to source dir (#3826) move source connectors to source/ Signed-off-by: tabVersion <tabvision@bupt.icu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3e72311 Author: Alex Chi <iskyzh@gmail.com> Date: Wed Jul 13 14:19:09 2022 +0800 feat(cli): add prof and tokio env config (#3824) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit b16c25a Author: Richard Chien <stdrc@outlook.com> Date: Wed Jul 13 13:42:40 2022 +0800 refactor(batch,agg): merge `Aggregator::update*` and refactor `SortAggExecutor` (#3794) * feed Aggregator at most one group at a time * impl new Aggregator functions for existing aggregators * impl `nth` for ArrayImplIterator to speed up `iter.skip(n)` * fix clippy warnings * add unittest for ArrayImplIterator::nth Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3f03b2e Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 12:39:16 2022 +0800 fix(type): fix decimal from_str and add e2e test (#3823) * chore(e2e): add e2e test for decimal special values * fix(type): fix decimal from_str and add e2e test * fix(type): add unit-test for decimal from_str commit eb1d49e Author: Richard Chien <stdrc@outlook.com> Date: Wed Jul 13 10:33:28 2022 +0800 feat(frontend): support order by clause of agg call (#3720) * support binding agg order by clause * support planning agg call with order by clause * print order by clause of agg call * correct the generation of BatchProject * publicly expose AggOrderBy::sort_exprs * remove a useless log * fix debug format of PlanAggCall * revert changes to PlanAggCall::filter * add plan tests for order by clause * return error for some invalid syntax * use Option<OrderType> instead of Option<bool> * move ORDER BY clause in debug output * fix plan tests * use Direction instead of OrderType * remove unused derive * change default value of AggOrderByExpr::direction to Direction::Asc * add a todo comment * change nulls_first field to bool without Option * fix plan tests Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ff4c0d7 Author: Steven Chua <stevengkc714@protonmail.com> Date: Wed Jul 13 09:56:01 2022 +0800 feat(frontend): Add planning and option in frontend for Lookup Join (#3763) * feat(frontend): Add plan node for batch lookup join * feat(frontend): Add config for using lookup join * feat(frontend): Change batch lookup join to unary node * feat(frontend): Add sources for lookup join node * feat(frontend): Fix bugs in Lookup Join * feat(frontend): Add error handling when data types of two sides of equality predicate are different * feat(frontend): Fix formatting and remove extraneous code * feat(frontend): fix bug * feat(frontend): Add e2e tests * feat(frontend): Fix e2e tests * feat(frontend): Better handling for when lookup join's conditions are not met * feat(frontend): Fix bug with comparing order key and predicate indices * feat(frontend): Refactored code and renamed config to rw_batch_enable_lookup_join * feat(frontend): Fix bug with output indices * feat(frontend): Fix test * feat(frontend): Fix another test * feat(frontend): Fix tests * feat(frontend): Change name and add more e2e tests * feat(frontend): Fix formatting * feat(frontend): Fix test bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 7a1331b Author: Li0k <yuli@singularity-data.com> Date: Tue Jul 12 21:19:00 2022 +0800 fix(utils): fix decimal flag bound (#3808) commit 6141051 Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Tue Jul 12 17:53:21 2022 +0800 feat(meta): add metrics in hummock manager lock and real process time (#3802) * feat(meta): add metrics in hummock manager lock and real process time * fix timer use * add lock type in metrics label * add grafana panel Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6dadbe0 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Tue Jul 12 17:13:08 2022 +0800 feat(StorageTableIter): parameterize on decoding (#3800) * refactor deserializer constructor * parameterize on deserializer * rerun ci Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 27a7ea8 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Tue Jul 12 16:48:12 2022 +0800 feat(sqlsmith): Generate EquiJoin (#3613) * generalize gen table * initial skeleton * pass columns * gen nested join factor * generate join_on expr * initialize join on binop * fmt * use inner join * clean * gen qualified column names * fmt * clean * clean * add planner test for join on and cross join * add expected logical plan * fix typo * disable equijoin * revert join.yaml Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ead67f7 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Tue Jul 12 16:36:24 2022 +0800 test(batch): add benchmark for sort merge join (#3797) * add gen_sorted_data * modify the visibility of modules to allow benchmarking * remove data_type of gen_data and gen_sorted_data * add benchmark for sort merge join Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 7b544ad Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Tue Jul 12 16:09:05 2022 +0800 refactor(storage): move hummock tests to a new crate to break crate cyclic dependency (#3795) * refactor(storage): move hummock tests to a new crate to break crate cyclic dependency * remove meta dependency for failpoint test * Update src/storage/hummock_test/src/vacuum_tests.rs Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add workspace-hack to the new craet Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6170ff2 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Tue Jul 12 15:51:20 2022 +0800 refactor(streaming): ensure correct storage table constructing for all streaming operators (#3765) * refactor * refactor look_up * fix * fix lookup * fix lookup * remove new_for_test * fix distribution of arrangement Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: Alex Chi <iskyzh@gmail.com> commit b618974 Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Tue Jul 12 15:32:31 2022 +0800 feat(executor): `DynamicFilter` write right table state exactly once - only if actor owns vnode = 0 (#3783) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci * use correct enum * add state table * fmt * prepare for cache policy * fmt * minor * minor * minor * Update src/stream/src/executor/managed_state/dynamic_filter.rs add license Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fmt * add state table for RHS * minor * fmt * add right_table to executor * minor * fmt * fix * clippy * fmt * fix double write * use correct * try to write on every msg * fix weird omissions * bump * minor * revert q18, q20 * revert q4 * remove inserts * only insert for right table * try to insert at barrier only * add asserts * debug * fix * fmt * only write on vnode = 0 * minor * minor * use bool as we need to read from RHS in the future... (for recovery)... Co-authored-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit fd0b7e2 Author: Bohan Zhang <tabvision@bupt.icu> Date: Tue Jul 12 14:26:12 2022 +0800 feat: kafka json append only sink (#3682) * stage Signed-off-by: tabVersion <tabvision@bupt.icu> * add datum_to_json_object Signed-off-by: Peng Chen <peng@singularity-data.com> * stage Signed-off-by: tabVersion <tabvision@bupt.icu> * finishing Signed-off-by: tabVersion <tabvision@bupt.icu> * make risedev happy Signed-off-by: tabVersion <tabvision@bupt.icu> * add ut Signed-off-by: tabVersion <tabvision@bupt.icu> * pending error: Transaction error: Operation not valid in state Ready Signed-off-by: tabVersion <tabvision@bupt.icu> * fix by changing epoch number Signed-off-by: tabVersion <tabvision@bupt.icu> * add doc for sink trait Signed-off-by: tabVersion <tabvision@bupt.icu> * add comment in exec Signed-off-by: tabVersion <tabvision@bupt.icu> * remove take_snapshot Signed-off-by: tabVersion <tabvision@bupt.icu> * fix warning Signed-off-by: tabVersion <tabvision@bupt.icu> * fix table_id Signed-off-by: tabVersion <tabvision@bupt.icu> Co-authored-by: Peng Chen <peng@singularity-data.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 0b65e68 Author: Alex Chi <iskyzh@gmail.com> Date: Tue Jul 12 14:00:53 2022 +0800 fix(dashboardv1): adapt new dispatcher design and add dynamic filter node (#3793) Signed-off-by: Alex Chi <iskyzh@gmail.com> commit 17bdf88 Author: TennyZhuang <zty0826@gmail.com> Date: Tue Jul 12 10:21:58 2022 +0800 build(deps): use official pulsar-rs instead of fork (#3787) Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8bd5f63 Author: TennyZhuang <zty0826@gmail.com> Date: Tue Jul 12 01:33:53 2022 +0800 fix(connector): support parsing non-string nexmark properties (#3784) * fix(connector): support parse non-string nexmark properties Signed-off-by: TennyZhuang <zty0826@gmail.com> * still use serde_json currently Signed-off-by: TennyZhuang <zty0826@gmail.com> * rollback to Box Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit f93d8e3 Author: xiangjinwu <17769960+xiangjinwu@users.noreply.github.com> Date: Tue Jul 12 01:07:05 2022 +0800 refactor: remove stale AggKind::RowCount (#3774) * refactor: remove stale AggKind::RowCount * update doc Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 2ee98b5 Author: StrikeW <wangsiyuanse@gmail.com> Date: Mon Jul 11 23:53:31 2022 +0800 fix(batch): fix order_by with struct table (#3782) * fix(batch): order_by with struct table * add more case Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit b7a969c Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Mon Jul 11 22:05:53 2022 +0800 feat(executor): Use `TableCatalog` and `StateTable` in `DynamicFilter` (only writes) (#3757) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci * use correct enum * add state table * fmt * prepare for cache policy * fmt * minor * minor * minor * Update src/stream/src/executor/managed_state/dynamic_filter.rs add license Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fmt * add state table for RHS * minor * fmt * add right_table to executor * minor * fmt * fix * clippy * fmt * fix double write * use correct * try to write on every msg * fix weird omissions Co-authored-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 5430e23 Author: Li0k <yuli@singularity-data.com> Date: Mon Jul 11 20:13:30 2022 +0800 chore(storage): remove value meta (#3779) commit 0608f0a Author: StrikeW <wangsiyuanse@gmail.com> Date: Mon Jul 11 17:36:36 2022 +0800 refactor(stream): refactor state of topN executor to one state table (#3747) commit 097e3eb Author: Bugen Zhao <i@bugenzhao.com> Date: Mon Jul 11 17:16:17 2022 +0800 feat(streaming): use multiple dispatcher for mview creation (#3758) * move mutation and barrier to stream_plan.proto Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add dispatcher when creating mview Signed-off-by: Bugen Zhao <i@bugenzhao.com> * from to proto of add dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * put actor info to shared context Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add dispatcher on compute node Signed-off-by: Bugen Zhao <i@bugenzhao.com> * clean up empty dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * cleanup Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix clippy Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix tests Signed-off-by: Bugen Zhao <i@bugenzhao.com> * use add dispatcher for source executor Signed-off-by: Bugen Zhao <i@bugenzhao.com> * rename dispatch to dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add comments Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix unique dispatcher id Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix it Signed-off-by: Bugen Zhao <i@bugenzhao.com> * use expect dead code Signed-off-by: Bugen Zhao <i@bugenzhao.com> commit 59579ce Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 16:58:54 2022 +0800 fix(batch): add first-match optimization for left semi join (#3773) add first-match optimization for left semi join commit 7d21848 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 16:09:01 2022 +0800 feat(common): add is_set to BitmapBuilder (#3768) add is_set to BitmapBuilder commit 78e5b66 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 15:33:31 2022 +0800 refactor(batch): say goodbye to the state machine in NLJ (#3724) * refactor(batch): say goodbye to the state machine * add some utilities and move right table to the outer loop * refactor * modify the unit test for left outer join to adapt to the semantical change * fix utility usage * remove unused imports * add some comments * more comments * fix format commit 7715827 Author: Zhanxiang (Patrick) Huang <hzxa21@hotmail.com> Date: Mon Jul 11 12:51:00 2022 +0800 refactor: remove vnode_mappings in CompactTask and GroupedSstableBuilder (#3764) * refactor: remove vnode_mappings in CompactTask * remove GroupedSstableBuilder Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit a8d0c57 Author: Xinhao Xu <84456268+xxhZs@users.noreply.github.com> Date: Mon Jul 11 12:10:40 2022 +0800 feat(common): move observer_manager to common service (#3754) * mv observer_manager to common_service * mv observer_manager to common_service * rename commit 33c3611 Author: zwang28 <70626450+zwang28@users.noreply.github.com> Date: Mon Jul 11 10:50:59 2022 +0800 fix: sync compaction group correctly (#3743) commit 00422c6 Author: Liang <44948473+soundOfDestiny@users.noreply.github.com> Date: Mon Jul 11 09:57:42 2022 +0800 perf(hummmock manager): remove unnecessary hummock version deltas (#3689) * perf(hummmock manager): remove unnecessary hummock version deltas * delete multiple version deltas in one trx * do not use vartrx for hummock version * do not use vartrx for hummockversion in commit_epoch * remove checkpoint_version_id Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 208ab51 Author: Alex Chi <iskyzh@gmail.com> Date: Sun Jul 10 16:38:39 2022 +0800 fix(risedev): unexpected rebuilds on Linux (#3761) * fix(risedev): unexpected rebuilds on Linux Signed-off-by: Alex Chi <iskyzh@gmail.com> * update workspace hack Signed-off-by: Alex Chi <iskyzh@gmail.com> * use bash instead of fish Co-authored-by: Bugen Zhao <i@bugenzhao.com> commit 301499f Author: Lee Zong Yu <65748142+marvenlee2486@users.noreply.github.com> Date: Sat Jul 9 14:51:38 2022 +0800 feat(metrics): Add Fragment Level Exchange Metrics (#3696) * Edit update.sh * Edit format error * Add payload to gitignore * Revert risedev.yml modification * Fix wrong gitignore * Fix wrong gitignore * Done coding for Fragment level exchange * Done coding for Fragment level exchange * Edit implementation error * Edit down fragment error * Delete println statement and revert configuration settings * Done coding for Fragment level exchange * Edit update.sh * Edit format error * Add payload to gitignore * Revert risedev.yml modification * Fix wrong gitignore * Done coding for Fragment level exchange * Edit implementation error * Edit down fragment error * Delete println statement and revert configuration settings * Edit Format and Resolve conflict * Resolve Test error * Resolve some error * generate dashboard * use rate_interval instead of 15s Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: Alex Chi <iskyzh@gmail.com> commit 92ee6d9 Author: Bugen Zhao <i@bugenzhao.com> Date: Fri Jul 8 23:25:12 2022 +0800 refactor(meta): improve create mview / scheduler readability (#3748) * refactor(meta): improve create mview / scheduler readability Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add some comments for hash mapping Signed-off-by: Bugen Zhao <i@bugenzhao.com> * refine docs Signed-off-by: Bugen Zhao <i@bugenzhao.com> * make yuanxin happy Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8fbe3f9 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Fri Jul 8 19:03:34 2022 +0800 refactor(streaming): use table catalog for materialize executor (#3742) * use catalog for materialize executor commit 23f41ab Author: August <pin@singularity-data.com> Date: Fri Jul 8 18:31:01 2022 +0800 feat: deprecate catalog ref id (#3745) feat: deprecate database/schema/table_ref_id Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit fa9e66c Author: Dylan <chenzl25@mail2.sysu.edu.cn> Date: Fri Jul 8 16:39:44 2022 +0800 feat(optimizer): support two phase topn and limit in batch mode (#3719) * feat(optimizer): support two phase topn and limit in batch mode * fix limit distribution in local batch mode * add test Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4b50016 Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Fri Jul 8 10:23:47 2022 +0200 fix(batch): fix batch scan distribution & handle more than one table scans in one stage (#3736) * fix batch scan distribution * handle more than one table scans in one stage Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 17c0417 Author: Yuhao Su <31772373+yuhao-su@users.noreply.github.com> Date: Fri Jul 8 14:25:09 2022 +0800 refactor(streaming): use table catalog in hash join (#3707) * use tablecatalog * fix infer_internal_table_catalog * cargo fix * fix tests * keys -> key * fix build executor * minor * fix incorrect order keys * fix comments * fix typo * add comment commit 2dc7992 Author: TennyZhuang <zty0826@gmail.com> Date: Fri Jul 8 14:12:58 2022 +0800 test(nexmark): add nexmark q10 q14 e2e test (#3740) * test(nexmark): add nexmark q10 e2e test Signed-off-by: TennyZhuang <zty0826@gmail.com> * test(nexmark): add nexmark q14 e2e test Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit d21b2b9 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Fri Jul 8 13:58:20 2022 +0800 chore(risedev): update grafana version (#3731) * update grafana version * enable auto update * "retry" commit aea1065 Author: Alex Chi <iskyzh@gmail.com> Date: Fri Jul 8 13:43:59 2022 +0800 fix(ci): use header fix action for license check (#3739) commit 7a86c4d Author: Tao Wu <wutao@singularity-data.com> Date: Fri Jul 8 10:30:39 2022 +0800 fix(sqlsmith): completely cover all exprs (#3737) commit a777217 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Thu Jul 7 22:44:39 2022 +0800 refactor(storage): remove cell_based serialize/deserialize from serde (#3722) * remove cell_based serialize/deserialize from serde commit 8d97480 Author: Tao Wu <wutao@singularity-data.com> Date: Thu Jul 7 22:23:34 2022 +0800 fix: support decimal mul interval (#3734) commit 9b64fca Author: Steven Chua <stevengkc714@protonmail.com> Date: Thu Jul 7 21:12:14 2022 +0800 fix(risedev): Revert Makefile.toml changes (#3732) Revert Makefile.toml changes Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 19cffce Author: Richard Chien <stdrc@outlook.com> Date: Thu Jul 7 20:57:15 2022 +0800 refactor(sqlparser): rename `parse_string_agg` to `parse_string_concat` (#3733) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit d767d99 Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Thu Jul 7 20:33:06 2022 +0800 feat(executor): Basic in-application-memory `DynamicFilterExecutor`, pass tpch q11, q22 e2e tests (#3681) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci Co-authored-by: TennyZhuang <zty0826@gmail.com> commit 07f282b Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 19:53:06 2022 +0800 chore(deps): bump dependencies (#3725) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 95586c8 Author: Yuhao Su <31772373+yuhao-su@users.noreply.github.com> Date: Thu Jul 7 19:02:54 2022 +0800 test(streaming): enable streaming e2e test for tpch q4, q18, q20 (#3718) enable e2e commit ca0f431 Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 18:11:07 2022 +0800 fix(metrics): use current actor id for backpressure (#3721) * fix(metrics): use current actor id for backpressure Signed-off-by: Alex Chi <iskyzh@gmail.com> * fix Signed-off-by: Alex Chi <iskyzh@gmail.com> commit 477a4be Author: August <pin@singularity-data.com> Date: Thu Jul 7 17:15:27 2022 +0800 fix: fix heartbeat req field and meta client impl (#3716) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit f9362b1 Author: Bowen <36908971+BowenXiao1999@users.noreply.github.com> Date: Thu Jul 7 17:03:13 2022 +0800 doc: add relational table schema doc + move files (#3712) * doc: add relational table schema doc + move files * fix comment * Update docs/relational_table/relational-table-schema.md Co-authored-by: congyi <58715567+wcy-fdu@users.noreply.github.com> Co-authored-by: congyi <58715567+wcy-fdu@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4a92eb6 Author: TennyZhuang <zty0826@gmail.com> Date: Thu Jul 7 16:50:54 2022 +0800 test(nexmark): explain q16 q17 (#3715) Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit e1fd06b Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Thu Jul 7 10:30:57 2022 +0200 feat(batch): prune scan partition according to scan_range (#3698) * prune vnode according to scan_range * disallow accessing other vnodes from batch scan * improve style * add ut * fix doc about dist key * fix Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ed4cb79 Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 16:11:15 2022 +0800 feat(storage): default block size = 1024 (#3463) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3bb55cf Author: zwang28 <70626450+zwang28@users.noreply.github.com> Date: Thu Jul 7 15:55:55 2022 +0800 fix: correctly implement check_state_consistency (#3711) commit 078c2f6 Author: xiangjinwu <17769960+xiangjinwu@users.noreply.github.com> Date: Thu Jul 7 15:35:38 2022 +0800 refactor(optimizer): cleanup LogicalAgg::prune_col (#3663) * refactor(optimizer): cleanup LogicalAgg::prune_col * planner test of prune-filter bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 82ef7ec Author: Tao Wu <wutao@singularity-data.com> Date: Thu Jul 7 14:54:18 2022 +0800 fix: reduce logging in the critical path (#3704) * fix: reduce logging in the critical path * fix cargo check Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 79eb45b Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Thu Jul 7 14:42:14 2022 +0800 feat(meta): add monitoring meta process metrics (#3706) * feat(meta): add monitoring meta process metrics * update procfs dependency * add libc dependency Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
mergify bot
added a commit
that referenced
this pull request
Aug 3, 2022
* fix do not schedule compact task Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * add unit aware compaction Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix strategy Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix strategy Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * tmp commit Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * remove unit id Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix meta Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix fmt Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix config Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix a large compaction Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * remove unit id Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * refactor overlap compaction picker Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * support sub level Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * refactor strategy Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * Squashed commit of the following: commit 048587d Author: lmatz <lmatz823@gmail.com> Date: Thu Jul 14 11:03:48 2022 +0800 fix(test): pass user name for regress tests (#3854) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 9b0d3a1 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Thu Jul 14 10:13:51 2022 +0800 feat(sqlsmith): generate group by (#3805) * implement sqlsmith group by * fmt * add required cols * pass valid cols to gen_col * gen col from valid_cols if available * override bound columns for group by * capture local column context * fix case where no FROM * fmt * cleanup * clean Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8231cf1 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Thu Jul 14 00:23:16 2022 +0800 feat(row-based encoding): introduce row-based encoding by using value encoding (#3835) * introduce row-based encoding 1.0 commit 1966c93 Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Wed Jul 13 17:08:09 2022 +0200 chore(proto): remove unused FilterScanNode (#3846) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 23388b9 Author: Wenzhuo Liu <lwzbill@foxmail.com> Date: Wed Jul 13 22:48:40 2022 +0800 feat(batch): support selective aggregation (#3683) * feat(batch): support selective aggregation for count * add e2e test for selective count * add group by test for selective count * remove option * add filter to approx count distinct * add general agg and fix approx count * add general distinct agg * format * add e2e test * add selective distinct agg e2e * avoid collect * fix count star * format * fix bug in optimizer Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 2a8a96d Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Wed Jul 13 16:07:03 2022 +0200 style(stream): exhaustive patterm for build_executor (#3848) commit ca49f6d Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 20:55:55 2022 +0800 feat(storage): support deserialize_prefix_len for prefix_bloom_filter (#3807) * feat(storage): support deserialize_prefix_len for prefix_bloom_filter * fix(storage): fix deserialize_prefix unit-test * chore(storage): add doc for encoding_data_size * fix(storage): fix encoding_data_size to return truely length in storage_encoding commit edc6660 Author: Dylan <chenzl25@mail2.sysu.edu.cn> Date: Wed Jul 13 20:38:23 2022 +0800 feat(frontend): support explain verbose (#3798) * feat(frontend) support explain verbose * prefix each field name with table_name * do-apply-planner-test * minor improvement * fix batch two phase agg which should satisfy all agg-call contain no distinct and order by * fix stream two phase agg which should satisfy all agg-call contain no distinct and order by * move explain_verbose flag from session config to optimizer context * fmt * fix test case commit 4a5503e Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 18:59:48 2022 +0800 fix(hash): fix hash_key_data_size (#3843) * fix(hash): fix hash_key_data_size * fix(hash): support Key8 for Boolean commit d0027c6 Author: Bugen Zhao <i@bugenzhao.com> Date: Wed Jul 13 17:53:04 2022 +0800 fix(test): datagen parallel e2e (#3840) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit aa09e44 Author: Huangjw <1223644280@qq.com> Date: Wed Jul 13 17:40:49 2022 +0800 chore(risedev): add external_labels and remote_write in prometeus config (#3825) * add remote write in prometheus config * fix config * add env PROMETHEUS_REMOTE_WRITE * cargo fmt * cargo fmt * update risedev yml * fix risedev yml * fix risedev yml Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6e45426 Author: Alex Chi <iskyzh@gmail.com> Date: Wed Jul 13 17:08:41 2022 +0800 chore(streaming): check epoch continuous (#3839) * chore(streaming): check epoch continuous Signed-off-by: Alex Chi <iskyzh@gmail.com> * fix unit test Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4fadf48 Author: Kaige Li <55606560+likg227@users.noreply.github.com> Date: Wed Jul 13 16:36:10 2022 +0800 feat(optimizer): distinct agg. (#3717) * feat(optimizer): distinct agg. * add comments. * add comments and tests. * correct planner test. * refactor. * improvement. * small fix. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 0f8148d Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Wed Jul 13 16:15:17 2022 +0800 feat(common): support generating `Datum` in field generator (#3827) * add generate_datum to generator traits * add MIN and MAX to OrderedF32 and OrderedF64 * make OrderedFloat serializable and expose impl_rand in common configs * implement generate_datum * add unit tests and fix format * fix bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4b511e8 Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Wed Jul 13 15:36:04 2022 +0800 feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap (#3680) * feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap * add docs Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 369b57e Author: Bohan Zhang <tabvision@bupt.icu> Date: Wed Jul 13 14:42:22 2022 +0800 refactor: move source connectors to source dir (#3826) move source connectors to source/ Signed-off-by: tabVersion <tabvision@bupt.icu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3e72311 Author: Alex Chi <iskyzh@gmail.com> Date: Wed Jul 13 14:19:09 2022 +0800 feat(cli): add prof and tokio env config (#3824) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit b16c25a Author: Richard Chien <stdrc@outlook.com> Date: Wed Jul 13 13:42:40 2022 +0800 refactor(batch,agg): merge `Aggregator::update*` and refactor `SortAggExecutor` (#3794) * feed Aggregator at most one group at a time * impl new Aggregator functions for existing aggregators * impl `nth` for ArrayImplIterator to speed up `iter.skip(n)` * fix clippy warnings * add unittest for ArrayImplIterator::nth Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3f03b2e Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 12:39:16 2022 +0800 fix(type): fix decimal from_str and add e2e test (#3823) * chore(e2e): add e2e test for decimal special values * fix(type): fix decimal from_str and add e2e test * fix(type): add unit-test for decimal from_str commit eb1d49e Author: Richard Chien <stdrc@outlook.com> Date: Wed Jul 13 10:33:28 2022 +0800 feat(frontend): support order by clause of agg call (#3720) * support binding agg order by clause * support planning agg call with order by clause * print order by clause of agg call * correct the generation of BatchProject * publicly expose AggOrderBy::sort_exprs * remove a useless log * fix debug format of PlanAggCall * revert changes to PlanAggCall::filter * add plan tests for order by clause * return error for some invalid syntax * use Option<OrderType> instead of Option<bool> * move ORDER BY clause in debug output * fix plan tests * use Direction instead of OrderType * remove unused derive * change default value of AggOrderByExpr::direction to Direction::Asc * add a todo comment * change nulls_first field to bool without Option * fix plan tests Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ff4c0d7 Author: Steven Chua <stevengkc714@protonmail.com> Date: Wed Jul 13 09:56:01 2022 +0800 feat(frontend): Add planning and option in frontend for Lookup Join (#3763) * feat(frontend): Add plan node for batch lookup join * feat(frontend): Add config for using lookup join * feat(frontend): Change batch lookup join to unary node * feat(frontend): Add sources for lookup join node * feat(frontend): Fix bugs in Lookup Join * feat(frontend): Add error handling when data types of two sides of equality predicate are different * feat(frontend): Fix formatting and remove extraneous code * feat(frontend): fix bug * feat(frontend): Add e2e tests * feat(frontend): Fix e2e tests * feat(frontend): Better handling for when lookup join's conditions are not met * feat(frontend): Fix bug with comparing order key and predicate indices * feat(frontend): Refactored code and renamed config to rw_batch_enable_lookup_join * feat(frontend): Fix bug with output indices * feat(frontend): Fix test * feat(frontend): Fix another test * feat(frontend): Fix tests * feat(frontend): Change name and add more e2e tests * feat(frontend): Fix formatting * feat(frontend): Fix test bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 7a1331b Author: Li0k <yuli@singularity-data.com> Date: Tue Jul 12 21:19:00 2022 +0800 fix(utils): fix decimal flag bound (#3808) commit 6141051 Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Tue Jul 12 17:53:21 2022 +0800 feat(meta): add metrics in hummock manager lock and real process time (#3802) * feat(meta): add metrics in hummock manager lock and real process time * fix timer use * add lock type in metrics label * add grafana panel Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6dadbe0 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Tue Jul 12 17:13:08 2022 +0800 feat(StorageTableIter): parameterize on decoding (#3800) * refactor deserializer constructor * parameterize on deserializer * rerun ci Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 27a7ea8 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Tue Jul 12 16:48:12 2022 +0800 feat(sqlsmith): Generate EquiJoin (#3613) * generalize gen table * initial skeleton * pass columns * gen nested join factor * generate join_on expr * initialize join on binop * fmt * use inner join * clean * gen qualified column names * fmt * clean * clean * add planner test for join on and cross join * add expected logical plan * fix typo * disable equijoin * revert join.yaml Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ead67f7 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Tue Jul 12 16:36:24 2022 +0800 test(batch): add benchmark for sort merge join (#3797) * add gen_sorted_data * modify the visibility of modules to allow benchmarking * remove data_type of gen_data and gen_sorted_data * add benchmark for sort merge join Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 7b544ad Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Tue Jul 12 16:09:05 2022 +0800 refactor(storage): move hummock tests to a new crate to break crate cyclic dependency (#3795) * refactor(storage): move hummock tests to a new crate to break crate cyclic dependency * remove meta dependency for failpoint test * Update src/storage/hummock_test/src/vacuum_tests.rs Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add workspace-hack to the new craet Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6170ff2 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Tue Jul 12 15:51:20 2022 +0800 refactor(streaming): ensure correct storage table constructing for all streaming operators (#3765) * refactor * refactor look_up * fix * fix lookup * fix lookup * remove new_for_test * fix distribution of arrangement Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: Alex Chi <iskyzh@gmail.com> commit b618974 Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Tue Jul 12 15:32:31 2022 +0800 feat(executor): `DynamicFilter` write right table state exactly once - only if actor owns vnode = 0 (#3783) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci * use correct enum * add state table * fmt * prepare for cache policy * fmt * minor * minor * minor * Update src/stream/src/executor/managed_state/dynamic_filter.rs add license Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fmt * add state table for RHS * minor * fmt * add right_table to executor * minor * fmt * fix * clippy * fmt * fix double write * use correct * try to write on every msg * fix weird omissions * bump * minor * revert q18, q20 * revert q4 * remove inserts * only insert for right table * try to insert at barrier only * add asserts * debug * fix * fmt * only write on vnode = 0 * minor * minor * use bool as we need to read from RHS in the future... (for recovery)... Co-authored-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit fd0b7e2 Author: Bohan Zhang <tabvision@bupt.icu> Date: Tue Jul 12 14:26:12 2022 +0800 feat: kafka json append only sink (#3682) * stage Signed-off-by: tabVersion <tabvision@bupt.icu> * add datum_to_json_object Signed-off-by: Peng Chen <peng@singularity-data.com> * stage Signed-off-by: tabVersion <tabvision@bupt.icu> * finishing Signed-off-by: tabVersion <tabvision@bupt.icu> * make risedev happy Signed-off-by: tabVersion <tabvision@bupt.icu> * add ut Signed-off-by: tabVersion <tabvision@bupt.icu> * pending error: Transaction error: Operation not valid in state Ready Signed-off-by: tabVersion <tabvision@bupt.icu> * fix by changing epoch number Signed-off-by: tabVersion <tabvision@bupt.icu> * add doc for sink trait Signed-off-by: tabVersion <tabvision@bupt.icu> * add comment in exec Signed-off-by: tabVersion <tabvision@bupt.icu> * remove take_snapshot Signed-off-by: tabVersion <tabvision@bupt.icu> * fix warning Signed-off-by: tabVersion <tabvision@bupt.icu> * fix table_id Signed-off-by: tabVersion <tabvision@bupt.icu> Co-authored-by: Peng Chen <peng@singularity-data.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 0b65e68 Author: Alex Chi <iskyzh@gmail.com> Date: Tue Jul 12 14:00:53 2022 +0800 fix(dashboardv1): adapt new dispatcher design and add dynamic filter node (#3793) Signed-off-by: Alex Chi <iskyzh@gmail.com> commit 17bdf88 Author: TennyZhuang <zty0826@gmail.com> Date: Tue Jul 12 10:21:58 2022 +0800 build(deps): use official pulsar-rs instead of fork (#3787) Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8bd5f63 Author: TennyZhuang <zty0826@gmail.com> Date: Tue Jul 12 01:33:53 2022 +0800 fix(connector): support parsing non-string nexmark properties (#3784) * fix(connector): support parse non-string nexmark properties Signed-off-by: TennyZhuang <zty0826@gmail.com> * still use serde_json currently Signed-off-by: TennyZhuang <zty0826@gmail.com> * rollback to Box Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit f93d8e3 Author: xiangjinwu <17769960+xiangjinwu@users.noreply.github.com> Date: Tue Jul 12 01:07:05 2022 +0800 refactor: remove stale AggKind::RowCount (#3774) * refactor: remove stale AggKind::RowCount * update doc Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 2ee98b5 Author: StrikeW <wangsiyuanse@gmail.com> Date: Mon Jul 11 23:53:31 2022 +0800 fix(batch): fix order_by with struct table (#3782) * fix(batch): order_by with struct table * add more case Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit b7a969c Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Mon Jul 11 22:05:53 2022 +0800 feat(executor): Use `TableCatalog` and `StateTable` in `DynamicFilter` (only writes) (#3757) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci * use correct enum * add state table * fmt * prepare for cache policy * fmt * minor * minor * minor * Update src/stream/src/executor/managed_state/dynamic_filter.rs add license Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fmt * add state table for RHS * minor * fmt * add right_table to executor * minor * fmt * fix * clippy * fmt * fix double write * use correct * try to write on every msg * fix weird omissions Co-authored-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 5430e23 Author: Li0k <yuli@singularity-data.com> Date: Mon Jul 11 20:13:30 2022 +0800 chore(storage): remove value meta (#3779) commit 0608f0a Author: StrikeW <wangsiyuanse@gmail.com> Date: Mon Jul 11 17:36:36 2022 +0800 refactor(stream): refactor state of topN executor to one state table (#3747) commit 097e3eb Author: Bugen Zhao <i@bugenzhao.com> Date: Mon Jul 11 17:16:17 2022 +0800 feat(streaming): use multiple dispatcher for mview creation (#3758) * move mutation and barrier to stream_plan.proto Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add dispatcher when creating mview Signed-off-by: Bugen Zhao <i@bugenzhao.com> * from to proto of add dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * put actor info to shared context Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add dispatcher on compute node Signed-off-by: Bugen Zhao <i@bugenzhao.com> * clean up empty dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * cleanup Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix clippy Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix tests Signed-off-by: Bugen Zhao <i@bugenzhao.com> * use add dispatcher for source executor Signed-off-by: Bugen Zhao <i@bugenzhao.com> * rename dispatch to dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add comments Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix unique dispatcher id Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix it Signed-off-by: Bugen Zhao <i@bugenzhao.com> * use expect dead code Signed-off-by: Bugen Zhao <i@bugenzhao.com> commit 59579ce Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 16:58:54 2022 +0800 fix(batch): add first-match optimization for left semi join (#3773) add first-match optimization for left semi join commit 7d21848 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 16:09:01 2022 +0800 feat(common): add is_set to BitmapBuilder (#3768) add is_set to BitmapBuilder commit 78e5b66 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 15:33:31 2022 +0800 refactor(batch): say goodbye to the state machine in NLJ (#3724) * refactor(batch): say goodbye to the state machine * add some utilities and move right table to the outer loop * refactor * modify the unit test for left outer join to adapt to the semantical change * fix utility usage * remove unused imports * add some comments * more comments * fix format commit 7715827 Author: Zhanxiang (Patrick) Huang <hzxa21@hotmail.com> Date: Mon Jul 11 12:51:00 2022 +0800 refactor: remove vnode_mappings in CompactTask and GroupedSstableBuilder (#3764) * refactor: remove vnode_mappings in CompactTask * remove GroupedSstableBuilder Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit a8d0c57 Author: Xinhao Xu <84456268+xxhZs@users.noreply.github.com> Date: Mon Jul 11 12:10:40 2022 +0800 feat(common): move observer_manager to common service (#3754) * mv observer_manager to common_service * mv observer_manager to common_service * rename commit 33c3611 Author: zwang28 <70626450+zwang28@users.noreply.github.com> Date: Mon Jul 11 10:50:59 2022 +0800 fix: sync compaction group correctly (#3743) commit 00422c6 Author: Liang <44948473+soundOfDestiny@users.noreply.github.com> Date: Mon Jul 11 09:57:42 2022 +0800 perf(hummmock manager): remove unnecessary hummock version deltas (#3689) * perf(hummmock manager): remove unnecessary hummock version deltas * delete multiple version deltas in one trx * do not use vartrx for hummock version * do not use vartrx for hummockversion in commit_epoch * remove checkpoint_version_id Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 208ab51 Author: Alex Chi <iskyzh@gmail.com> Date: Sun Jul 10 16:38:39 2022 +0800 fix(risedev): unexpected rebuilds on Linux (#3761) * fix(risedev): unexpected rebuilds on Linux Signed-off-by: Alex Chi <iskyzh@gmail.com> * update workspace hack Signed-off-by: Alex Chi <iskyzh@gmail.com> * use bash instead of fish Co-authored-by: Bugen Zhao <i@bugenzhao.com> commit 301499f Author: Lee Zong Yu <65748142+marvenlee2486@users.noreply.github.com> Date: Sat Jul 9 14:51:38 2022 +0800 feat(metrics): Add Fragment Level Exchange Metrics (#3696) * Edit update.sh * Edit format error * Add payload to gitignore * Revert risedev.yml modification * Fix wrong gitignore * Fix wrong gitignore * Done coding for Fragment level exchange * Done coding for Fragment level exchange * Edit implementation error * Edit down fragment error * Delete println statement and revert configuration settings * Done coding for Fragment level exchange * Edit update.sh * Edit format error * Add payload to gitignore * Revert risedev.yml modification * Fix wrong gitignore * Done coding for Fragment level exchange * Edit implementation error * Edit down fragment error * Delete println statement and revert configuration settings * Edit Format and Resolve conflict * Resolve Test error * Resolve some error * generate dashboard * use rate_interval instead of 15s Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: Alex Chi <iskyzh@gmail.com> commit 92ee6d9 Author: Bugen Zhao <i@bugenzhao.com> Date: Fri Jul 8 23:25:12 2022 +0800 refactor(meta): improve create mview / scheduler readability (#3748) * refactor(meta): improve create mview / scheduler readability Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add some comments for hash mapping Signed-off-by: Bugen Zhao <i@bugenzhao.com> * refine docs Signed-off-by: Bugen Zhao <i@bugenzhao.com> * make yuanxin happy Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8fbe3f9 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Fri Jul 8 19:03:34 2022 +0800 refactor(streaming): use table catalog for materialize executor (#3742) * use catalog for materialize executor commit 23f41ab Author: August <pin@singularity-data.com> Date: Fri Jul 8 18:31:01 2022 +0800 feat: deprecate catalog ref id (#3745) feat: deprecate database/schema/table_ref_id Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit fa9e66c Author: Dylan <chenzl25@mail2.sysu.edu.cn> Date: Fri Jul 8 16:39:44 2022 +0800 feat(optimizer): support two phase topn and limit in batch mode (#3719) * feat(optimizer): support two phase topn and limit in batch mode * fix limit distribution in local batch mode * add test Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4b50016 Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Fri Jul 8 10:23:47 2022 +0200 fix(batch): fix batch scan distribution & handle more than one table scans in one stage (#3736) * fix batch scan distribution * handle more than one table scans in one stage Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 17c0417 Author: Yuhao Su <31772373+yuhao-su@users.noreply.github.com> Date: Fri Jul 8 14:25:09 2022 +0800 refactor(streaming): use table catalog in hash join (#3707) * use tablecatalog * fix infer_internal_table_catalog * cargo fix * fix tests * keys -> key * fix build executor * minor * fix incorrect order keys * fix comments * fix typo * add comment commit 2dc7992 Author: TennyZhuang <zty0826@gmail.com> Date: Fri Jul 8 14:12:58 2022 +0800 test(nexmark): add nexmark q10 q14 e2e test (#3740) * test(nexmark): add nexmark q10 e2e test Signed-off-by: TennyZhuang <zty0826@gmail.com> * test(nexmark): add nexmark q14 e2e test Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit d21b2b9 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Fri Jul 8 13:58:20 2022 +0800 chore(risedev): update grafana version (#3731) * update grafana version * enable auto update * "retry" commit aea1065 Author: Alex Chi <iskyzh@gmail.com> Date: Fri Jul 8 13:43:59 2022 +0800 fix(ci): use header fix action for license check (#3739) commit 7a86c4d Author: Tao Wu <wutao@singularity-data.com> Date: Fri Jul 8 10:30:39 2022 +0800 fix(sqlsmith): completely cover all exprs (#3737) commit a777217 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Thu Jul 7 22:44:39 2022 +0800 refactor(storage): remove cell_based serialize/deserialize from serde (#3722) * remove cell_based serialize/deserialize from serde commit 8d97480 Author: Tao Wu <wutao@singularity-data.com> Date: Thu Jul 7 22:23:34 2022 +0800 fix: support decimal mul interval (#3734) commit 9b64fca Author: Steven Chua <stevengkc714@protonmail.com> Date: Thu Jul 7 21:12:14 2022 +0800 fix(risedev): Revert Makefile.toml changes (#3732) Revert Makefile.toml changes Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 19cffce Author: Richard Chien <stdrc@outlook.com> Date: Thu Jul 7 20:57:15 2022 +0800 refactor(sqlparser): rename `parse_string_agg` to `parse_string_concat` (#3733) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit d767d99 Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Thu Jul 7 20:33:06 2022 +0800 feat(executor): Basic in-application-memory `DynamicFilterExecutor`, pass tpch q11, q22 e2e tests (#3681) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci Co-authored-by: TennyZhuang <zty0826@gmail.com> commit 07f282b Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 19:53:06 2022 +0800 chore(deps): bump dependencies (#3725) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 95586c8 Author: Yuhao Su <31772373+yuhao-su@users.noreply.github.com> Date: Thu Jul 7 19:02:54 2022 +0800 test(streaming): enable streaming e2e test for tpch q4, q18, q20 (#3718) enable e2e commit ca0f431 Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 18:11:07 2022 +0800 fix(metrics): use current actor id for backpressure (#3721) * fix(metrics): use current actor id for backpressure Signed-off-by: Alex Chi <iskyzh@gmail.com> * fix Signed-off-by: Alex Chi <iskyzh@gmail.com> commit 477a4be Author: August <pin@singularity-data.com> Date: Thu Jul 7 17:15:27 2022 +0800 fix: fix heartbeat req field and meta client impl (#3716) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit f9362b1 Author: Bowen <36908971+BowenXiao1999@users.noreply.github.com> Date: Thu Jul 7 17:03:13 2022 +0800 doc: add relational table schema doc + move files (#3712) * doc: add relational table schema doc + move files * fix comment * Update docs/relational_table/relational-table-schema.md Co-authored-by: congyi <58715567+wcy-fdu@users.noreply.github.com> Co-authored-by: congyi <58715567+wcy-fdu@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4a92eb6 Author: TennyZhuang <zty0826@gmail.com> Date: Thu Jul 7 16:50:54 2022 +0800 test(nexmark): explain q16 q17 (#3715) Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit e1fd06b Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Thu Jul 7 10:30:57 2022 +0200 feat(batch): prune scan partition according to scan_range (#3698) * prune vnode according to scan_range * disallow accessing other vnodes from batch scan * improve style * add ut * fix doc about dist key * fix Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ed4cb79 Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 16:11:15 2022 +0800 feat(storage): default block size = 1024 (#3463) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3bb55cf Author: zwang28 <70626450+zwang28@users.noreply.github.com> Date: Thu Jul 7 15:55:55 2022 +0800 fix: correctly implement check_state_consistency (#3711) commit 078c2f6 Author: xiangjinwu <17769960+xiangjinwu@users.noreply.github.com> Date: Thu Jul 7 15:35:38 2022 +0800 refactor(optimizer): cleanup LogicalAgg::prune_col (#3663) * refactor(optimizer): cleanup LogicalAgg::prune_col * planner test of prune-filter bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 82ef7ec Author: Tao Wu <wutao@singularity-data.com> Date: Thu Jul 7 14:54:18 2022 +0800 fix: reduce logging in the critical path (#3704) * fix: reduce logging in the critical path * fix cargo check Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 79eb45b Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Thu Jul 7 14:42:14 2022 +0800 feat(meta): add monitoring meta process metrics (#3706) * feat(meta): add monitoring meta process metrics * update procfs dependency * add libc dependency Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix format Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix large compaction Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * merge e2e_test from main Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * simple compact Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * refactor splits Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix splits Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix bug Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix unit test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * tmp commit Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test bug Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix version Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix bug Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix clippy warn Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix format Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * split tier compaction file Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix license Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix trivial move Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix compact Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix compactor Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix commit Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
nasnoisaac
pushed a commit
to nasnoisaac/risingwave
that referenced
this pull request
Aug 9, 2022
nasnoisaac
pushed a commit
to nasnoisaac/risingwave
that referenced
this pull request
Aug 9, 2022
* fix do not schedule compact task Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * add unit aware compaction Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix strategy Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix strategy Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * tmp commit Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * remove unit id Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix meta Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix fmt Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix config Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix a large compaction Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * remove unit id Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * refactor overlap compaction picker Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * support sub level Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * refactor strategy Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * Squashed commit of the following: commit 048587d Author: lmatz <lmatz823@gmail.com> Date: Thu Jul 14 11:03:48 2022 +0800 fix(test): pass user name for regress tests (risingwavelabs#3854) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 9b0d3a1 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Thu Jul 14 10:13:51 2022 +0800 feat(sqlsmith): generate group by (risingwavelabs#3805) * implement sqlsmith group by * fmt * add required cols * pass valid cols to gen_col * gen col from valid_cols if available * override bound columns for group by * capture local column context * fix case where no FROM * fmt * cleanup * clean Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8231cf1 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Thu Jul 14 00:23:16 2022 +0800 feat(row-based encoding): introduce row-based encoding by using value encoding (risingwavelabs#3835) * introduce row-based encoding 1.0 commit 1966c93 Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Wed Jul 13 17:08:09 2022 +0200 chore(proto): remove unused FilterScanNode (risingwavelabs#3846) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 23388b9 Author: Wenzhuo Liu <lwzbill@foxmail.com> Date: Wed Jul 13 22:48:40 2022 +0800 feat(batch): support selective aggregation (risingwavelabs#3683) * feat(batch): support selective aggregation for count * add e2e test for selective count * add group by test for selective count * remove option * add filter to approx count distinct * add general agg and fix approx count * add general distinct agg * format * add e2e test * add selective distinct agg e2e * avoid collect * fix count star * format * fix bug in optimizer Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 2a8a96d Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Wed Jul 13 16:07:03 2022 +0200 style(stream): exhaustive patterm for build_executor (risingwavelabs#3848) commit ca49f6d Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 20:55:55 2022 +0800 feat(storage): support deserialize_prefix_len for prefix_bloom_filter (risingwavelabs#3807) * feat(storage): support deserialize_prefix_len for prefix_bloom_filter * fix(storage): fix deserialize_prefix unit-test * chore(storage): add doc for encoding_data_size * fix(storage): fix encoding_data_size to return truely length in storage_encoding commit edc6660 Author: Dylan <chenzl25@mail2.sysu.edu.cn> Date: Wed Jul 13 20:38:23 2022 +0800 feat(frontend): support explain verbose (risingwavelabs#3798) * feat(frontend) support explain verbose * prefix each field name with table_name * do-apply-planner-test * minor improvement * fix batch two phase agg which should satisfy all agg-call contain no distinct and order by * fix stream two phase agg which should satisfy all agg-call contain no distinct and order by * move explain_verbose flag from session config to optimizer context * fmt * fix test case commit 4a5503e Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 18:59:48 2022 +0800 fix(hash): fix hash_key_data_size (risingwavelabs#3843) * fix(hash): fix hash_key_data_size * fix(hash): support Key8 for Boolean commit d0027c6 Author: Bugen Zhao <i@bugenzhao.com> Date: Wed Jul 13 17:53:04 2022 +0800 fix(test): datagen parallel e2e (risingwavelabs#3840) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit aa09e44 Author: Huangjw <1223644280@qq.com> Date: Wed Jul 13 17:40:49 2022 +0800 chore(risedev): add external_labels and remote_write in prometeus config (risingwavelabs#3825) * add remote write in prometheus config * fix config * add env PROMETHEUS_REMOTE_WRITE * cargo fmt * cargo fmt * update risedev yml * fix risedev yml * fix risedev yml Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6e45426 Author: Alex Chi <iskyzh@gmail.com> Date: Wed Jul 13 17:08:41 2022 +0800 chore(streaming): check epoch continuous (risingwavelabs#3839) * chore(streaming): check epoch continuous Signed-off-by: Alex Chi <iskyzh@gmail.com> * fix unit test Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4fadf48 Author: Kaige Li <55606560+likg227@users.noreply.github.com> Date: Wed Jul 13 16:36:10 2022 +0800 feat(optimizer): distinct agg. (risingwavelabs#3717) * feat(optimizer): distinct agg. * add comments. * add comments and tests. * correct planner test. * refactor. * improvement. * small fix. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 0f8148d Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Wed Jul 13 16:15:17 2022 +0800 feat(common): support generating `Datum` in field generator (risingwavelabs#3827) * add generate_datum to generator traits * add MIN and MAX to OrderedF32 and OrderedF64 * make OrderedFloat serializable and expose impl_rand in common configs * implement generate_datum * add unit tests and fix format * fix bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4b511e8 Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Wed Jul 13 15:36:04 2022 +0800 feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap (risingwavelabs#3680) * feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap * add docs Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 369b57e Author: Bohan Zhang <tabvision@bupt.icu> Date: Wed Jul 13 14:42:22 2022 +0800 refactor: move source connectors to source dir (risingwavelabs#3826) move source connectors to source/ Signed-off-by: tabVersion <tabvision@bupt.icu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3e72311 Author: Alex Chi <iskyzh@gmail.com> Date: Wed Jul 13 14:19:09 2022 +0800 feat(cli): add prof and tokio env config (risingwavelabs#3824) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit b16c25a Author: Richard Chien <stdrc@outlook.com> Date: Wed Jul 13 13:42:40 2022 +0800 refactor(batch,agg): merge `Aggregator::update*` and refactor `SortAggExecutor` (risingwavelabs#3794) * feed Aggregator at most one group at a time * impl new Aggregator functions for existing aggregators * impl `nth` for ArrayImplIterator to speed up `iter.skip(n)` * fix clippy warnings * add unittest for ArrayImplIterator::nth Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3f03b2e Author: Li0k <yuli@singularity-data.com> Date: Wed Jul 13 12:39:16 2022 +0800 fix(type): fix decimal from_str and add e2e test (risingwavelabs#3823) * chore(e2e): add e2e test for decimal special values * fix(type): fix decimal from_str and add e2e test * fix(type): add unit-test for decimal from_str commit eb1d49e Author: Richard Chien <stdrc@outlook.com> Date: Wed Jul 13 10:33:28 2022 +0800 feat(frontend): support order by clause of agg call (risingwavelabs#3720) * support binding agg order by clause * support planning agg call with order by clause * print order by clause of agg call * correct the generation of BatchProject * publicly expose AggOrderBy::sort_exprs * remove a useless log * fix debug format of PlanAggCall * revert changes to PlanAggCall::filter * add plan tests for order by clause * return error for some invalid syntax * use Option<OrderType> instead of Option<bool> * move ORDER BY clause in debug output * fix plan tests * use Direction instead of OrderType * remove unused derive * change default value of AggOrderByExpr::direction to Direction::Asc * add a todo comment * change nulls_first field to bool without Option * fix plan tests Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ff4c0d7 Author: Steven Chua <stevengkc714@protonmail.com> Date: Wed Jul 13 09:56:01 2022 +0800 feat(frontend): Add planning and option in frontend for Lookup Join (risingwavelabs#3763) * feat(frontend): Add plan node for batch lookup join * feat(frontend): Add config for using lookup join * feat(frontend): Change batch lookup join to unary node * feat(frontend): Add sources for lookup join node * feat(frontend): Fix bugs in Lookup Join * feat(frontend): Add error handling when data types of two sides of equality predicate are different * feat(frontend): Fix formatting and remove extraneous code * feat(frontend): fix bug * feat(frontend): Add e2e tests * feat(frontend): Fix e2e tests * feat(frontend): Better handling for when lookup join's conditions are not met * feat(frontend): Fix bug with comparing order key and predicate indices * feat(frontend): Refactored code and renamed config to rw_batch_enable_lookup_join * feat(frontend): Fix bug with output indices * feat(frontend): Fix test * feat(frontend): Fix another test * feat(frontend): Fix tests * feat(frontend): Change name and add more e2e tests * feat(frontend): Fix formatting * feat(frontend): Fix test bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 7a1331b Author: Li0k <yuli@singularity-data.com> Date: Tue Jul 12 21:19:00 2022 +0800 fix(utils): fix decimal flag bound (risingwavelabs#3808) commit 6141051 Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Tue Jul 12 17:53:21 2022 +0800 feat(meta): add metrics in hummock manager lock and real process time (risingwavelabs#3802) * feat(meta): add metrics in hummock manager lock and real process time * fix timer use * add lock type in metrics label * add grafana panel Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6dadbe0 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Tue Jul 12 17:13:08 2022 +0800 feat(StorageTableIter): parameterize on decoding (risingwavelabs#3800) * refactor deserializer constructor * parameterize on deserializer * rerun ci Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 27a7ea8 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Tue Jul 12 16:48:12 2022 +0800 feat(sqlsmith): Generate EquiJoin (risingwavelabs#3613) * generalize gen table * initial skeleton * pass columns * gen nested join factor * generate join_on expr * initialize join on binop * fmt * use inner join * clean * gen qualified column names * fmt * clean * clean * add planner test for join on and cross join * add expected logical plan * fix typo * disable equijoin * revert join.yaml Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ead67f7 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Tue Jul 12 16:36:24 2022 +0800 test(batch): add benchmark for sort merge join (risingwavelabs#3797) * add gen_sorted_data * modify the visibility of modules to allow benchmarking * remove data_type of gen_data and gen_sorted_data * add benchmark for sort merge join Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 7b544ad Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Tue Jul 12 16:09:05 2022 +0800 refactor(storage): move hummock tests to a new crate to break crate cyclic dependency (risingwavelabs#3795) * refactor(storage): move hummock tests to a new crate to break crate cyclic dependency * remove meta dependency for failpoint test * Update src/storage/hummock_test/src/vacuum_tests.rs Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add workspace-hack to the new craet Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 6170ff2 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Tue Jul 12 15:51:20 2022 +0800 refactor(streaming): ensure correct storage table constructing for all streaming operators (risingwavelabs#3765) * refactor * refactor look_up * fix * fix lookup * fix lookup * remove new_for_test * fix distribution of arrangement Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: Alex Chi <iskyzh@gmail.com> commit b618974 Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Tue Jul 12 15:32:31 2022 +0800 feat(executor): `DynamicFilter` write right table state exactly once - only if actor owns vnode = 0 (risingwavelabs#3783) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci * use correct enum * add state table * fmt * prepare for cache policy * fmt * minor * minor * minor * Update src/stream/src/executor/managed_state/dynamic_filter.rs add license Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fmt * add state table for RHS * minor * fmt * add right_table to executor * minor * fmt * fix * clippy * fmt * fix double write * use correct * try to write on every msg * fix weird omissions * bump * minor * revert q18, q20 * revert q4 * remove inserts * only insert for right table * try to insert at barrier only * add asserts * debug * fix * fmt * only write on vnode = 0 * minor * minor * use bool as we need to read from RHS in the future... (for recovery)... Co-authored-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit fd0b7e2 Author: Bohan Zhang <tabvision@bupt.icu> Date: Tue Jul 12 14:26:12 2022 +0800 feat: kafka json append only sink (risingwavelabs#3682) * stage Signed-off-by: tabVersion <tabvision@bupt.icu> * add datum_to_json_object Signed-off-by: Peng Chen <peng@singularity-data.com> * stage Signed-off-by: tabVersion <tabvision@bupt.icu> * finishing Signed-off-by: tabVersion <tabvision@bupt.icu> * make risedev happy Signed-off-by: tabVersion <tabvision@bupt.icu> * add ut Signed-off-by: tabVersion <tabvision@bupt.icu> * pending error: Transaction error: Operation not valid in state Ready Signed-off-by: tabVersion <tabvision@bupt.icu> * fix by changing epoch number Signed-off-by: tabVersion <tabvision@bupt.icu> * add doc for sink trait Signed-off-by: tabVersion <tabvision@bupt.icu> * add comment in exec Signed-off-by: tabVersion <tabvision@bupt.icu> * remove take_snapshot Signed-off-by: tabVersion <tabvision@bupt.icu> * fix warning Signed-off-by: tabVersion <tabvision@bupt.icu> * fix table_id Signed-off-by: tabVersion <tabvision@bupt.icu> Co-authored-by: Peng Chen <peng@singularity-data.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 0b65e68 Author: Alex Chi <iskyzh@gmail.com> Date: Tue Jul 12 14:00:53 2022 +0800 fix(dashboardv1): adapt new dispatcher design and add dynamic filter node (risingwavelabs#3793) Signed-off-by: Alex Chi <iskyzh@gmail.com> commit 17bdf88 Author: TennyZhuang <zty0826@gmail.com> Date: Tue Jul 12 10:21:58 2022 +0800 build(deps): use official pulsar-rs instead of fork (risingwavelabs#3787) Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8bd5f63 Author: TennyZhuang <zty0826@gmail.com> Date: Tue Jul 12 01:33:53 2022 +0800 fix(connector): support parsing non-string nexmark properties (risingwavelabs#3784) * fix(connector): support parse non-string nexmark properties Signed-off-by: TennyZhuang <zty0826@gmail.com> * still use serde_json currently Signed-off-by: TennyZhuang <zty0826@gmail.com> * rollback to Box Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit f93d8e3 Author: xiangjinwu <17769960+xiangjinwu@users.noreply.github.com> Date: Tue Jul 12 01:07:05 2022 +0800 refactor: remove stale AggKind::RowCount (risingwavelabs#3774) * refactor: remove stale AggKind::RowCount * update doc Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 2ee98b5 Author: StrikeW <wangsiyuanse@gmail.com> Date: Mon Jul 11 23:53:31 2022 +0800 fix(batch): fix order_by with struct table (risingwavelabs#3782) * fix(batch): order_by with struct table * add more case Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit b7a969c Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Mon Jul 11 22:05:53 2022 +0800 feat(executor): Use `TableCatalog` and `StateTable` in `DynamicFilter` (only writes) (risingwavelabs#3757) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci * use correct enum * add state table * fmt * prepare for cache policy * fmt * minor * minor * minor * Update src/stream/src/executor/managed_state/dynamic_filter.rs add license Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fmt * add state table for RHS * minor * fmt * add right_table to executor * minor * fmt * fix * clippy * fmt * fix double write * use correct * try to write on every msg * fix weird omissions Co-authored-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 5430e23 Author: Li0k <yuli@singularity-data.com> Date: Mon Jul 11 20:13:30 2022 +0800 chore(storage): remove value meta (risingwavelabs#3779) commit 0608f0a Author: StrikeW <wangsiyuanse@gmail.com> Date: Mon Jul 11 17:36:36 2022 +0800 refactor(stream): refactor state of topN executor to one state table (risingwavelabs#3747) commit 097e3eb Author: Bugen Zhao <i@bugenzhao.com> Date: Mon Jul 11 17:16:17 2022 +0800 feat(streaming): use multiple dispatcher for mview creation (risingwavelabs#3758) * move mutation and barrier to stream_plan.proto Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add dispatcher when creating mview Signed-off-by: Bugen Zhao <i@bugenzhao.com> * from to proto of add dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * put actor info to shared context Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add dispatcher on compute node Signed-off-by: Bugen Zhao <i@bugenzhao.com> * clean up empty dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * cleanup Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix clippy Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix tests Signed-off-by: Bugen Zhao <i@bugenzhao.com> * use add dispatcher for source executor Signed-off-by: Bugen Zhao <i@bugenzhao.com> * rename dispatch to dispatcher Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add comments Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix unique dispatcher id Signed-off-by: Bugen Zhao <i@bugenzhao.com> * fix it Signed-off-by: Bugen Zhao <i@bugenzhao.com> * use expect dead code Signed-off-by: Bugen Zhao <i@bugenzhao.com> commit 59579ce Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 16:58:54 2022 +0800 fix(batch): add first-match optimization for left semi join (risingwavelabs#3773) add first-match optimization for left semi join commit 7d21848 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 16:09:01 2022 +0800 feat(common): add is_set to BitmapBuilder (risingwavelabs#3768) add is_set to BitmapBuilder commit 78e5b66 Author: wzzzzd <108382774+wzzzzd@users.noreply.github.com> Date: Mon Jul 11 15:33:31 2022 +0800 refactor(batch): say goodbye to the state machine in NLJ (risingwavelabs#3724) * refactor(batch): say goodbye to the state machine * add some utilities and move right table to the outer loop * refactor * modify the unit test for left outer join to adapt to the semantical change * fix utility usage * remove unused imports * add some comments * more comments * fix format commit 7715827 Author: Zhanxiang (Patrick) Huang <hzxa21@hotmail.com> Date: Mon Jul 11 12:51:00 2022 +0800 refactor: remove vnode_mappings in CompactTask and GroupedSstableBuilder (risingwavelabs#3764) * refactor: remove vnode_mappings in CompactTask * remove GroupedSstableBuilder Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit a8d0c57 Author: Xinhao Xu <84456268+xxhZs@users.noreply.github.com> Date: Mon Jul 11 12:10:40 2022 +0800 feat(common): move observer_manager to common service (risingwavelabs#3754) * mv observer_manager to common_service * mv observer_manager to common_service * rename commit 33c3611 Author: zwang28 <70626450+zwang28@users.noreply.github.com> Date: Mon Jul 11 10:50:59 2022 +0800 fix: sync compaction group correctly (risingwavelabs#3743) commit 00422c6 Author: Liang <44948473+soundOfDestiny@users.noreply.github.com> Date: Mon Jul 11 09:57:42 2022 +0800 perf(hummmock manager): remove unnecessary hummock version deltas (risingwavelabs#3689) * perf(hummmock manager): remove unnecessary hummock version deltas * delete multiple version deltas in one trx * do not use vartrx for hummock version * do not use vartrx for hummockversion in commit_epoch * remove checkpoint_version_id Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 208ab51 Author: Alex Chi <iskyzh@gmail.com> Date: Sun Jul 10 16:38:39 2022 +0800 fix(risedev): unexpected rebuilds on Linux (risingwavelabs#3761) * fix(risedev): unexpected rebuilds on Linux Signed-off-by: Alex Chi <iskyzh@gmail.com> * update workspace hack Signed-off-by: Alex Chi <iskyzh@gmail.com> * use bash instead of fish Co-authored-by: Bugen Zhao <i@bugenzhao.com> commit 301499f Author: Lee Zong Yu <65748142+marvenlee2486@users.noreply.github.com> Date: Sat Jul 9 14:51:38 2022 +0800 feat(metrics): Add Fragment Level Exchange Metrics (risingwavelabs#3696) * Edit update.sh * Edit format error * Add payload to gitignore * Revert risedev.yml modification * Fix wrong gitignore * Fix wrong gitignore * Done coding for Fragment level exchange * Done coding for Fragment level exchange * Edit implementation error * Edit down fragment error * Delete println statement and revert configuration settings * Done coding for Fragment level exchange * Edit update.sh * Edit format error * Add payload to gitignore * Revert risedev.yml modification * Fix wrong gitignore * Done coding for Fragment level exchange * Edit implementation error * Edit down fragment error * Delete println statement and revert configuration settings * Edit Format and Resolve conflict * Resolve Test error * Resolve some error * generate dashboard * use rate_interval instead of 15s Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: Alex Chi <iskyzh@gmail.com> commit 92ee6d9 Author: Bugen Zhao <i@bugenzhao.com> Date: Fri Jul 8 23:25:12 2022 +0800 refactor(meta): improve create mview / scheduler readability (risingwavelabs#3748) * refactor(meta): improve create mview / scheduler readability Signed-off-by: Bugen Zhao <i@bugenzhao.com> * add some comments for hash mapping Signed-off-by: Bugen Zhao <i@bugenzhao.com> * refine docs Signed-off-by: Bugen Zhao <i@bugenzhao.com> * make yuanxin happy Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 8fbe3f9 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Fri Jul 8 19:03:34 2022 +0800 refactor(streaming): use table catalog for materialize executor (risingwavelabs#3742) * use catalog for materialize executor commit 23f41ab Author: August <pin@singularity-data.com> Date: Fri Jul 8 18:31:01 2022 +0800 feat: deprecate catalog ref id (risingwavelabs#3745) feat: deprecate database/schema/table_ref_id Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit fa9e66c Author: Dylan <chenzl25@mail2.sysu.edu.cn> Date: Fri Jul 8 16:39:44 2022 +0800 feat(optimizer): support two phase topn and limit in batch mode (risingwavelabs#3719) * feat(optimizer): support two phase topn and limit in batch mode * fix limit distribution in local batch mode * add test Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4b50016 Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Fri Jul 8 10:23:47 2022 +0200 fix(batch): fix batch scan distribution & handle more than one table scans in one stage (risingwavelabs#3736) * fix batch scan distribution * handle more than one table scans in one stage Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 17c0417 Author: Yuhao Su <31772373+yuhao-su@users.noreply.github.com> Date: Fri Jul 8 14:25:09 2022 +0800 refactor(streaming): use table catalog in hash join (risingwavelabs#3707) * use tablecatalog * fix infer_internal_table_catalog * cargo fix * fix tests * keys -> key * fix build executor * minor * fix incorrect order keys * fix comments * fix typo * add comment commit 2dc7992 Author: TennyZhuang <zty0826@gmail.com> Date: Fri Jul 8 14:12:58 2022 +0800 test(nexmark): add nexmark q10 q14 e2e test (risingwavelabs#3740) * test(nexmark): add nexmark q10 e2e test Signed-off-by: TennyZhuang <zty0826@gmail.com> * test(nexmark): add nexmark q14 e2e test Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit d21b2b9 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Fri Jul 8 13:58:20 2022 +0800 chore(risedev): update grafana version (risingwavelabs#3731) * update grafana version * enable auto update * "retry" commit aea1065 Author: Alex Chi <iskyzh@gmail.com> Date: Fri Jul 8 13:43:59 2022 +0800 fix(ci): use header fix action for license check (risingwavelabs#3739) commit 7a86c4d Author: Tao Wu <wutao@singularity-data.com> Date: Fri Jul 8 10:30:39 2022 +0800 fix(sqlsmith): completely cover all exprs (risingwavelabs#3737) commit a777217 Author: congyi <58715567+wcy-fdu@users.noreply.github.com> Date: Thu Jul 7 22:44:39 2022 +0800 refactor(storage): remove cell_based serialize/deserialize from serde (risingwavelabs#3722) * remove cell_based serialize/deserialize from serde commit 8d97480 Author: Tao Wu <wutao@singularity-data.com> Date: Thu Jul 7 22:23:34 2022 +0800 fix: support decimal mul interval (risingwavelabs#3734) commit 9b64fca Author: Steven Chua <stevengkc714@protonmail.com> Date: Thu Jul 7 21:12:14 2022 +0800 fix(risedev): Revert Makefile.toml changes (risingwavelabs#3732) Revert Makefile.toml changes Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 19cffce Author: Richard Chien <stdrc@outlook.com> Date: Thu Jul 7 20:57:15 2022 +0800 refactor(sqlparser): rename `parse_string_agg` to `parse_string_concat` (risingwavelabs#3733) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit d767d99 Author: jon-chuang <9093549+jon-chuang@users.noreply.github.com> Date: Thu Jul 7 20:33:06 2022 +0800 feat(executor): Basic in-application-memory `DynamicFilterExecutor`, pass tpch q11, q22 e2e tests (risingwavelabs#3681) * initial * stash * do-appply-planner-test * fmt * clippy * license * minor * stash * fix required dist, output indices * fmt * stash * revert * do-apply-planner-test * success! * more e2e * minor cleanup * stash * fix * fix * fix * minor * minor * madsim * clippy * fmt * minor * fix * fix, apply code review suggestions * fix * fix, improve * simplify * clippy * refactor into sub functions * clippy * minor * fix * fix: rowsort * fmt * use literal in expression * clippy * fmt * use madsim btreemap * minor * fmt * respect previous behaviour - return empty message * bump-ci Co-authored-by: TennyZhuang <zty0826@gmail.com> commit 07f282b Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 19:53:06 2022 +0800 chore(deps): bump dependencies (risingwavelabs#3725) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 95586c8 Author: Yuhao Su <31772373+yuhao-su@users.noreply.github.com> Date: Thu Jul 7 19:02:54 2022 +0800 test(streaming): enable streaming e2e test for tpch q4, q18, q20 (risingwavelabs#3718) enable e2e commit ca0f431 Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 18:11:07 2022 +0800 fix(metrics): use current actor id for backpressure (risingwavelabs#3721) * fix(metrics): use current actor id for backpressure Signed-off-by: Alex Chi <iskyzh@gmail.com> * fix Signed-off-by: Alex Chi <iskyzh@gmail.com> commit 477a4be Author: August <pin@singularity-data.com> Date: Thu Jul 7 17:15:27 2022 +0800 fix: fix heartbeat req field and meta client impl (risingwavelabs#3716) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit f9362b1 Author: Bowen <36908971+BowenXiao1999@users.noreply.github.com> Date: Thu Jul 7 17:03:13 2022 +0800 doc: add relational table schema doc + move files (risingwavelabs#3712) * doc: add relational table schema doc + move files * fix comment * Update docs/relational_table/relational-table-schema.md Co-authored-by: congyi <58715567+wcy-fdu@users.noreply.github.com> Co-authored-by: congyi <58715567+wcy-fdu@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 4a92eb6 Author: TennyZhuang <zty0826@gmail.com> Date: Thu Jul 7 16:50:54 2022 +0800 test(nexmark): explain q16 q17 (risingwavelabs#3715) Signed-off-by: TennyZhuang <zty0826@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit e1fd06b Author: xxchan <37948597+xxchan@users.noreply.github.com> Date: Thu Jul 7 10:30:57 2022 +0200 feat(batch): prune scan partition according to scan_range (risingwavelabs#3698) * prune vnode according to scan_range * disallow accessing other vnodes from batch scan * improve style * add ut * fix doc about dist key * fix Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit ed4cb79 Author: Alex Chi <iskyzh@gmail.com> Date: Thu Jul 7 16:11:15 2022 +0800 feat(storage): default block size = 1024 (risingwavelabs#3463) Signed-off-by: Alex Chi <iskyzh@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 3bb55cf Author: zwang28 <70626450+zwang28@users.noreply.github.com> Date: Thu Jul 7 15:55:55 2022 +0800 fix: correctly implement check_state_consistency (risingwavelabs#3711) commit 078c2f6 Author: xiangjinwu <17769960+xiangjinwu@users.noreply.github.com> Date: Thu Jul 7 15:35:38 2022 +0800 refactor(optimizer): cleanup LogicalAgg::prune_col (risingwavelabs#3663) * refactor(optimizer): cleanup LogicalAgg::prune_col * planner test of prune-filter bug Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 82ef7ec Author: Tao Wu <wutao@singularity-data.com> Date: Thu Jul 7 14:54:18 2022 +0800 fix: reduce logging in the critical path (risingwavelabs#3704) * fix: reduce logging in the critical path * fix cargo check Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 79eb45b Author: William Wen <44139337+wenym1@users.noreply.github.com> Date: Thu Jul 7 14:42:14 2022 +0800 feat(meta): add monitoring meta process metrics (risingwavelabs#3706) * feat(meta): add monitoring meta process metrics * update procfs dependency * add libc dependency Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix format Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix large compaction Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * merge e2e_test from main Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * simple compact Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * refactor splits Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix splits Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix bug Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix unit test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * tmp commit Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test bug Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix version Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix bug Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix clippy warn Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix format Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * split tier compaction file Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix license Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix trivial move Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix compact Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix compactor Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix commit Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix conflict Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> * fix test Signed-off-by: Little-Wallace <bupt2013211450@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
Now we can run sqlsmith e2e against RisingWave without failures. Sqlsmith still lacks many features and the testing is not always stable. So I will include it in our CI process when I think it's stable enough.
Checklist
./risedev check(or alias,./risedev c)Release note
Please create a release note for your changes. In the release note, focus on the impact on users, and mention the environment or conditions where the impact may occur.
Refer to a related PR or issue link (optional)
#2571
Fixed #3341