v1.2.0-nightly

What's Changed

Exciting New Features ✨

• feat(SOUNDS LIKE): add "SOUNDS LIKE" syntax for String comparing. by @neil4dong in #11043
• feat(function): support window function percent_rank. by @youngsofun in #11094
• feat: support parse varbinary as String Type by @sundy-li in #11108
• feat: accelerate dphyp algo by @xudong963 in #11078
• feat: support inject cancel point into remove stage by @sundy-li in #11119
• feat: Allow load region from env for S3 by @Xuanwo in #11127
• feat: add timeout in shutdown service by @sundy-li in #11128
• feat: consider date type in histogram by @xudong963 in #11135
• feat(meta-service): Split log store and state-machine store by @drmingdrmer in #11124
• feat(storage): replace support cluster stats generator by @zhyass in #11129
• feat: support agg state combinator by @sundy-li in #11143
• feat: improve selectivity by updating statistics by @xudong963 in #11151
• feat(query): table meta optimize by @jun0315 in #11015
• feat(meta-service): write data version to local store by @drmingdrmer in #11160
• feat: lazy materialize according to virtual column _row_id. by @RinChanNOWWW in #11088
• feat(function): Support json_path_query using JSON path by @b41sh in #11142
• feat(expression): add try cast adaptor by @sundy-li in #11171
• feat: Add community for DocPages by @Chasen-Zhang in #11203
• feat: improve cardinality estimate by @xudong963 in #11175
• feat(query): Create table allows column default value using a expression by @b41sh in #11193
• feat: add bitmap data type by @ariesdevil in #11097
• feat(query): add optimize hints SET_VAR by @TCeason in #11208
• feat(query): tools for decoding the new table meta data by @jun0315 in #11234
• feat(query): new hash table and parallel finalize for hash join by @Dousir9 in #11140
• feat: flight sql server enable tcp no delay by @sundy-li in #11252
• feat(query): show columns support display view engine by @TCeason in #11257
• feat: add Enterprise license with bootstrap implementation by @ZhiHanZ in #11153
• feat(query): efficiently memory for multiple group key by @zhang2014 in #11110
• feat: covert like expression to comparison by @xudong963 in #11262
• feat(parser): allows escape stage location by @flaneur2020 in #11266
• feat: prune partitions by internal column predicates. by @RinChanNOWWW in #11248
• feat(parser): support ANSI quoted identifier by @flaneur2020 in #11247
• feat: add like domain by @sundy-li in #11265
• feat: add license parsing function by @ZhiHanZ in #11303
• feat: add bitmap_count function by @sundy-li in #11304
• feat: support to use config override parquet_fast_read_bytes and max_storage_io_requests settings by @ZhiHanZ in #11305
• feat: support alias in where clause. by @RinChanNOWWW in #11272
• feat: add vacuum table SQL by @lichuang in #11308
• feat: view can be created on stage table query. by @RinChanNOWWW in #11314
• docs: add license faq by @ZhiHanZ in #11317
• feat: optimize join by reducing blocking at probe phase by @xudong963 in #11321
• feat: move table_test_fixture to test_kits crate by @lichuang in #11322
• feat: move gc::utils to test_kits crate by @lichuang in #11340
• feat(query): Allow insert invalid JSON by @b41sh in #11345
• feat(meta): track the time spent on meta client methods by @drmingdrmer in #11364
• feat: validates the value of max_storage_io_requests being by @Mehrbod2002 in #11367
• feat(meta): add match_seq field to TxnDeleteRequest by @drmingdrmer in #11356
• feat: add vacuum impl by @lichuang in #11320
• feat: Bump OpenDAL to 0.34.0 by @Xuanwo in #11376
• feat(schema-api): truncate_file: delete copied file in small chunks by @drmingdrmer in #11388
• feat: optimize broadcast join by @xudong963 in #11365
• feat: add vacuum dry run support by @lichuang in #11392
• feat: copy into table with columns. by @youngsofun in #11371
• feat: add Databend enterprise license key by @ZhiHanZ in #11395
• feat: bind databend into python lib by @sundy-li in #11354
• feat(query): Optimize read system.columns data by @TCeason in #11398
• feat(query): try optimize create table by @TCeason in #11427
• feat(python): add Dataframe & SessionContext by @sundy-li in #11420
• feat: Adding EE tip for docs by @Chasen-Zhang in #11434
• feat(python): fix context by @sundy-li in #11435
• feat(query): optimize alter and create view when has lots of tables by @TCeason in #11445
• feat(query): add func date_format and str_to_date and str_to_timestamp by @TCeason in #11442
• feat: fix ee integration tests by @ZhiHanZ in #11448
• feat(python): support tenant by @sundy-li in #11452
• feat(meta): add replication state to meta service API /v1/cluster/status by @drmingdrmer in #11457
• feat(planner): remove trivial cast by @Dousir9 in #11421
• feat(query): Support Computed columns by @b41sh in #11391
• feat(query): make date_format throw err by @TCeason in #11455
• feat(query): add create/drop index support by @ariesdevil in #11414
• feat(query): improve decimal div decimal by @sundy-li in #11464
• feat: support copy from URI with transform. by @youngsofun in #11467
• feat: auto query rewriting for aggregation index. by @RinChanNOWWW in #11418
• feat: add 5 bitmap scalar functions by @silver-ymz in #11487
• feat(query): optimize show tables with pushdown filter only contain eq database by @TCeason in #11490
• feat(meta-service): Upgrade Databend on-disk Data Version system by @drmingdrmer in #11479
• feat: add bitmap logical operation functions by @gitccl in #11507
• feat(query): improve cardinality estimate by @Dousir9 in #11394
• feat(front): add FunctionDescription component by @Chasen-Zhang in #11518
• feat: support replace with stage attachment by @ZhiHanZ in #11519
• feat(python): support configuration from env by @sundy-li in #11529
• feat: support intdiv operator // by @gitccl in #11527
• feat: add system.indexes and show indexes command. by @RinChanNOWWW in #11520
• feat: add table lock for compact by @zhyass in #11238
• feat: add bitmap count aggregate function by @gitccl in #11542
• feat: add sub_bitmap by @Mehrbod2002 in #11540
• feat(window function): Add window function lag and lead by @ariesdevil in #11244
• feat: push down aggregating index info to physcial scan. by @RinChanNOWWW in #11489
• feat(storage): Add native COS support by @Xuanwo in #11559
• feat: create\drop\desc data mask ddl by @lichuang in #11545
• feat: bitmap_subset_limit and bitmap_subset_in_range by @Mehrbod2002 in #11556
• feat: support azure openai api by @BohuTANG in #11572
• feat: support IEJoin by @xudong963 in #11412
• feat(FlightSQL): should not use session ID as token by @jun0315 in #11588
• feat: support more than two predicates for iejoin by @xudong963 in #11584
• feat: add window function first_value and last_value by @gitccl in #11587
• feat: new window function nth_value. by @RinChanNOWWW in #11600
• feat(planner): Integrate DPhyp algorithm with CascadesOptimizer by @Dousir9 in #11586
• feat(query): add range function & aliases in select list by @sundy-li in #11621
• feat: add alter table column set data mask policy support by @lichuang in #11599
• feat(front search): Add AI searching by @Chasen-Zhang in #11633
• feat: Copy into support optimize hint by @jun0315 in #11614
• feat(query): Support generate virtual columns by @b41sh in #11590
• feat(query): insert/mutation/update deduplicate based on label by @jun0315 in #11610
• feat: support distributed Top-N. by @RinChanNOWWW in #11662
• feat(query): Allow root user alter password by @b41sh in #11671
• feat(query): enable default lazy_topn_threshold setting to 1000 by @sundy-li in #11677
• feat: Bump OpenDAL to 0.37.0 by @Xuanwo in #11683
• feat(meta): add seq to ExpireValue by @drmingdrmer in #11686
• feat: support column position like $N. by @youngsofun in #11672
• feat: select from stage support NDJson. by @youngsofun in #11701
• feat: allow using built-in format name when select from stage. by @youngsofun in #11709
• feat(query): Support computed columns by @b41sh in #11689
• feat(query): add dataframe by @TCeason in #11632
• feat(query): add some columns into information_schema.tables by @TCeason in #11721
• feat: support merge join by @xudong963 in #11669
• refactor(query): improve left and right join by @Dousir9 in #11682
• feat: port comparable row format from Apache Arrow to Databend. by @RinChanNOWWW in #11712
• feat: kill http query after timeout. by @youngsofun in #11727
• feat: support delete with subquery by @xudong963 in #11714
• feat: add number_of_blocks,number_of_segments for system.tables by @JackTan25 in #11740
• feat: make explain delete work by @xudong963 in #11741
• feat: block_per_segment check by @JackTan25 in #11749
• feat: support more subqueries for delete statement by @xudong963 in #11747
• feat(query): Support add/drop computed columns by @b41sh in #11745
• feat: select $<column_position> from CSV by @youngsofun in #11739
• feat(storage): compact support clustering table by @zhyass in #11742
• feat: Alter set options by @JackTan25 in #11746
• feat(executor): Introduce QueryProfileManager to collect query profilings by @leiysky in #11760
• feat: support subquery in update statement by @xudong963 in #11763
• feat: implement Background service for databend by @ZhiHanZ in #11751
• feat: Integrate with icelake for iceberg support by @Xuanwo in #11785
• feat: alter table to rename column by @lichuang in #11786
• feat: select $<column_position> from TSV. by @youngsofun in #11792
• feat(query): Variant support row sort by @b41sh in #11765
• feat: add system.background_jobs and system.background_tasks table by @ZhiHanZ in #11798
• feat(query): unnest function support variant data type by @b41sh in #11801
• feat: http query deduplication api by @akoshchiy in #11787
• feat(storage): add optional create_on for segment block meta by @ariesdevil in #11808
• feat: CSV allow at most 1000 extra fields. by @youngsofun in #11814
• feat(cli): Better box display by @TCeason in #11756
• feat: enforce copy CSV on_error=continue by @youngsofun in #11816
• feat: fine tune for background compaction by @ZhiHanZ in #11815
• feat: streaming load support on_error=continue by @youngsofun in #11819
• feat(agg index): initial impl refresh index by @ariesdevil in #11634
• feat: support manually execute jobs by @ZhiHanZ in #11829
• feat: introduce distributed deletion by @SkyFan2002 in #11777
• feat(python): support register table functions by @sundy-li in #11841
• feat(python): support mac os pypi crates by @sundy-li in #11844
• feat(planner): add tpch explain to sqllogictest by @Dousir9 in #11832
• feat(python): support mac os pypi crates2 by @sundy-li in #11847
• feat(python): add macos bindings3 by @sundy-li in #11853
• feat: scan data from aggregating index. by @RinChanNOWWW in #11561
• feat(query): reduce the memory usage and memory copy of hash join by @Dousir9 in #11859
• feat: support to limit feature on license by @ZhiHanZ in #11879
• feat(query): Disable arbitrary hostnames when creating users by @b41sh in #11818
• feat(planner): support more operators for dphyp by @Dousir9 in #11802
• feat(iceberg): Implement read_partitions for IcebergTable by @Xuanwo in #11897
• feat: Implement Long run test scripts by @ZhiHanZ in #11869
• feat: Distributed copy into V1 by @JackTan25 in #11840
• feat: get_table_history filter only drop tables by @lichuang in #11903
• feat: add show drop table DDL by @lichuang in #11880

Thoughtful Bug Fix 🔧

• fix(query): fix insert value parse string to json by @b41sh in #11080
• fix: cannot share a db which is created from share, and test cases by @lichuang in #11081
• fix: unknown share table hits of share tnc7yee14.ontime_shared_2 on DROP TABLE by @flaneur2020 in #11095
• fix(storage): read snapshot with wrong version by @zhyass in #11099
• fix: Highlight blog code by @Chasen-Zhang in #11104
• fix(copy): fix copy abort not in query_log. by @youngsofun in #11102
• fix: join cardinality estimate by @xudong963 in #11105
• fix: iframe video style by @Chasen-Zhang in #11112
• fix: Storage Service OSS's env is not loaded by @Xuanwo in #11113
• fix: copy from parquet oom by @youngsofun in #11123
• fix(query): fix topk pruner bug by @sundy-li in #11154
• fix(meta-upgrade-09): revert mistakenly removed testing file meta-v23.txt by @drmingdrmer in #11163
• fix: Fix COS could return "Invalid Argument" while starting by @Xuanwo in #11184
• fix: Fix OSS query sign with security token by @Xuanwo in #11201
• fix: the map build_segment_id_map returns should contain format ver by @dantengsky in #11212
• fix: fix purge test bugs by @lichuang in #11210
• fix: support Top-N lazy materialization for native format. by @RinChanNOWWW in #11190
• fix: fix row id generation for native format with cluster key. by @RinChanNOWWW in #11227
• fix: stateless tests maybe failure sometimes by @zhyass in #11239
• fix: expire_at in upsert_copied_files_request by @zhyass in #11241
• fix: permission error on DROP USER with account_admin by @flaneur2020 in #11228
• fix(query): filter hintsuffix need check the EOI token by @TCeason in #11246
• fix: copy small parquet error on s3. by @youngsofun in #11256
• fix: use TableScheme::project to create the replace plan's schema by @dantengsky in #11258
• fix(ci): add order by to fix flaky test by @TCeason in #11261
• fix: page reader should use num_values of chunk. by @youngsofun in #11271
• fix(ci): comment a test by @TCeason in #11274
• fix(create): fix table options validation by @JackTan25 in #11290
• fix(query): fix insert JSON value in csv file by @b41sh in #11307
• fix: truncate should commit new meta to meta-server first by @dantengsky in #11319
• fix(planner): remove obsolete EOI token in insert probing by @leiysky in #11325
• fix(query): fix partial insert select by @sundy-li in #11330
• fix: fix test_kits release mod build fail by @lichuang in #11337
• fix(cluster): add max message size config for server by @zhang2014 in #11346
• fix(meta): set max encode/decode size to 16MB for meta-grpc and raft client/server by @drmingdrmer in #11343
• fix: enable kill query before pipeline started. by @youngsofun in #11336
• fix(query): QueryCtx add query_settings item to fix sql hint set_var affect session settings by @TCeason in #11360
• fix(meta): upgrade openraft to 0.8.4-alpha.3; bug fix, improve performance, etc by @drmingdrmer in #11402
• fix(styles): Adding menu overflow-y by @Chasen-Zhang in #11406
• fix(http): Don't print whole SQL for insert by @Xuanwo in #11409
• fix(storage): update/delete oom by @zhyass in #11386
• fix: oom when load CSV with many columns. by @youngsofun in #11413
• fix(planner): correctly resolve aggregate plan input type by @andylokandy in #11349
• fix: intervals_to_bitmap() wrong assert. by @youngsofun in #11451
• fix: use operator.scan instead of list. by @youngsofun in #11450
• fix(cluster): close channel when network error by @zhang2014 in #11407
• fix(query): fix SRF as arg of other function by @b41sh in #11475
• fix: do_vacuum should use retention_time to call purge funtion; fix test fail by @lichuang in #11478
• fix: support 64K pagesize on aarch64 [skip ci] by @everpcpc in #11484
• fix: code: 4000, message: invalid data (tag for enum is not valid, found 7): compaction error by @dantengsky in #11480
• fix: fix list indexes and add tests by @ariesdevil in #11488
• fix(query): fix show err with special name by @TCeason in #11497
• fix: fix lazy topn on native format. by @RinChanNOWWW in #11504
• fix: modify duplicated test case for SOUNDS LIKE syntax by @Jake-00 in #11503
• fix(python): add tests by @sundy-li in #11505
• fix: lost DataBlocks when copy with parquet_fast_read_bytes. by @youngsofun in #11511
• fix(python): add tests by @sundy-li in #11506
• fix(query): fix create table if not exists by @sundy-li in #11495
• fix(styles): FunctionDescription styles by @Chasen-Zhang in #11522
• fix(query): fix use db err because of not normalize ident by @TCeason in #11492
• fix: multi-cols pruning of replace into by @dantengsky in #11558
• fix: fix drop data mask bug by @lichuang in #11579
• fix: fix lag/lead default value when column is nullable by @ariesdevil in #11589
• fix: MergeIntoOperationAggregator should run data block deserialization task on dedicated thread by @dantengsky in #11597
• fix: support named window in expression. by @RinChanNOWWW in #11602
• fix(query): cannot unset global setting if has same session setting by @zhang2014 in #11627
• fix: support expressions in window function. by @RinChanNOWWW in #11613
• fix: AskBend request url by @Chasen-Zhang in #11636
• fix(front): search styles by @Chasen-Zhang in #11640
• fix: cached items should be kept being evict, till enough room be made for the new item. by @lichuang in #11620
• fix: iejoin hang by @xudong963 in #11642
• fix: fix hash join hang by @xudong963 in #11654
• fix(query): fix parse json with escaped and UTF-8 characters by @b41sh in #11657
• fix(storage/s3): Add allow_anonymous support for storage by @Xuanwo in #11661
• fix(query): json cast to string as escaped string by @b41sh in #11665
• fix(query): incorrect block index for lazy topn by @zhang2014 in #11692
• fix(query): Fix json special and foreign characters display errors by @b41sh in #11698
• fix: fix code logic by @JackTan25 in #11716
• fix: try to cast to target data type when check_number. by @RinChanNOWWW in #11720
• chore(sqllogictests): fix column types of MySQLClient by @jonahgao in #11718
• fix: incorrect query_key in systems.query_log for insert into ... with attachment queries by @JackTan25 in #11732
• fix: copy duplicate data when split large block. by @youngsofun in #11755
• fix(storage): recluster panic by @zhyass in #11762
• fix(query): fix ShortSql func by @sundy-li in #11767
• fix(test): The hint syntax must follow immediately after an COPY keyword by @TCeason in #11757
• fix: Config not loaded after bump serde by @Xuanwo in #11782
• fix(unit-test): Update MACOSX_DEPLOYMENT_TARGET value by @akoshchiy in #11783
• fix: refactor and fix internal columns binding. by @RinChanNOWWW in #11776
• fix(query): to_date(timestamp) result is equal with timestamp date part by @TCeason in #11768
• fix: copy on_error use wrong num_rows to pop column builder. by @youngsofun in #11807
• fix(planner): check unicode to prevent invalid string index access by @leiysky in #11811
• fix(query): consider DST when parse string to ts by @TCeason in #11794
• fix: sanitize the headers in request log by @flaneur2020 in #11817
• fix(query): fix dead lock for join by @zhang2014 in #11826
• fix: consider lazy columns when pruning unused columns. by @RinChanNOWWW in #11833
• fix: deletion with sub query by @dantengsky in #11834
• fix(query): parse string ignores unescape \" by @b41sh in #11839
• fix: only cache query result from fuse table. by @RinChanNOWWW in #11843
• fix(query): revert take compact by @Dousir9 in #11848
• fix(query): dataframe sort api need bind projection by @TCeason in #11849
• fix: union's schema type is invalid by @xudong963 in #11871
• fix: rows fetcher should complete the block if the input is empty. by @RinChanNOWWW in #11891
• fix: copy from parquet get wrong TS from Int96. by @youngsofun in #11899

Code Refactor 🎉

• refactor: revert to using all endpoints if all are unhealthy. by @drmingdrmer in #11111
• refactor(meta-service): handle every error when joining into a cluster by @drmingdrmer in #11144
• refactor(storage): optimize purge by @zhyass in #11017
• refactor(query): avoid OOM when select from ParquetTable. by @youngsofun in #11165
• refactor(copy): rewrite copy into table for parquet. by @youngsofun in #11173
• refactor: table meta format_version by @dantengsky in #11211
• refactor: optimize reading small parquet. by @youngsofun in #11231
• refactor: refactor order by clause analyzing and binding. by @RinChanNOWWW in #11233
• refactor(optimizer): add constant folding by @andylokandy in #11216
• refactor(schema-api): refactor remove_table_copied_files by @drmingdrmer in #11351
• refactor(schema-api): remove unused upsert_table_copied_file_info from trait SchemaApi and trait Database by @drmingdrmer in #11379
• refactor: hybrid segment cache by @dantengsky in #11347
• refactor(meta): remove UpsertTableCopiedFileReq.table_id by @drmingdrmer in #11381
• refactor: remove binary meta-upgrade-09 by @drmingdrmer in #11416
• refactor(query): remove segment id by @zhang2014 in #11486
• refactor(query): refactor segment pruner by @zhang2014 in #11510
• refactor(storage): make range indexer support default column by @sundy-li in #11554
• refactor: optimize data block processing in MergeIntoOperationAggregator by @dantengsky in #11571
• refactor: unify copy pipeline. by @youngsofun in #11569
• refactor: switch table meta encoding scheme to msgpack by @dantengsky in #11592
• refactor(storage): unify commit path of table mutation and append by @zhyass in #11626
• refactor: add number of segment and block to common_meta_app::schema::TableStatistics by @dantengsky in #11476
• refactor: improve single pipeline merge sort for Top-N. by @RinChanNOWWW in #11681
• refactor: Remove HTTP timeout after OpenDAL timeout layer added by @Xuanwo in #11707
• refactor: optimize decoding timestamp column. by @youngsofun in #11750
• refactor: optimize pruning phase of the delete statement by @dantengsky in #11759
• refactor(executor): parallel async tasks for async transform by @zhang2014 in #11770
• refactor(meta-service): when starting up, retry registering node by @drmingdrmer in #11779
• refactor: optimize replace into data block reading by @dantengsky in #11780
• refactor(query): remove enable_distributed_eval_index setting by @zhang2014 in #11790
• refactor: copy use BlockMetaInfo to pass RowBatch. by @youngsofun in #11856
• refactor(meta): start() does not block, retry until ready by @drmingdrmer in #11864
• refactor: optimize on conflict key digesting of stmt replace into by @dantengsky in #11885

Build/Testing/CI Infra Changes 🔌

• ci(function): test explain window function. by @youngsofun in #11077
• ci: use larger runner for hive release build [skip ci] by @everpcpc in #11082
• ci: add basic sqllogic test before release binaries [skip ci] by @everpcpc in #11117
• ci: tmp allow failure for release sqllogictest [skip ci] by @everpcpc in #11130
• ci: fix continue on error for linux release by @everpcpc in #11134
• ci: fix static link with libstdc++ for musl build by @everpcpc in #11145
• ci: gather docker logs on upload failure by @everpcpc in #11179
• ci: fix upload failure for stateful test cluster [skip ci] by @everpcpc in #11188
• ci: only pack .databend for upload failure [skip ci] by @everpcpc in #11192
• ci(benchmark): update to use new bendsql by @everpcpc in #11224
• ci: do not ignore error on benchmark & release [skip ci] by @everpcpc in #11226
• ci: add cache for cargo registry by @everpcpc in #11243
• ci: use bendsql --time for benchmark [skip ci] by @everpcpc in #11273
• ci: use BUILD_PROFILE for artifact download/upload by @everpcpc in #11279
• ci: optimize build workflows by @everpcpc in #11285
• ci: include metactl in release by @everpcpc in #11291
• ci: Add rust-analyzer in rust toolchain by @Xuanwo in #11299
• ci: fix install dependency for stateless test by @everpcpc in #11315
• ci: change benchmark machine name for cloud by @everpcpc in #11327
• ci: run tpch benchmark on Medium by @everpcpc in #11331
• ci: fetch only needed binaries for test by @everpcpc in #11341
• ci: fix workflow benchmark in release [skip ci] by @everpcpc in #11348
• ci: fix make binary executable for github artifacts by @everpcpc in #11352
• ci: fix concurrency group for benchmark with head_ref [skip ci] by @everpcpc in #11372
• ci: fix download artifacts for sqllogic test macos [skip ci] by @everpcpc in #11377
• ci: compress debug sections with flags by @everpcpc in #11401
• ci: test & release python binding by @everpcpc in #11431
• ci: better check to run ee test by @everpcpc in #11437
• ci: fix inputs tag for python binding [skip ci] by @everpcpc in #11439
• ci: fix publish python binding to pypi by @everpcpc in #11440
• ci: do not release source for python binding [skip ci] by @everpcpc in #11466
• ci: fix ignore path [skip ci] by @everpcpc in #11470
• ci: add install_lapack in dev_setup.sh by @SkyFan2002 in #11539
• ci: checkout merge commit on benchmark by @everpcpc in #11550
• ci: fix checkout for benchmark in pr by @everpcpc in #11560
• ci: run sqllogic ydb with 8c16g by @everpcpc in #11637
• ci: run production ci on self-hosted by @everpcpc in #11639
• ci: fix artifacts for release build by @everpcpc in #11643
• ci: fix syntax error in production ci by @everpcpc in #11644
• ci: fix build profile for test script metactl restore by @everpcpc in #11646
• ci: fix test condition for ee by @everpcpc in #11647
• ci: enable sccache for macos build by @everpcpc in #11648
• ci: fix warehouse size for benchmark by @everpcpc in #11724
• ci: Bump rust to latest toolchain by @Xuanwo in #11711
• ci: build image for pr label ci-cloud by @everpcpc in #11875
• ci: fix build cloud docker image [skip ci] by @everpcpc in #11877
• ci: disable crates cache [skip ci] by @everpcpc in #11878
• ci: add workflow for pr by @everpcpc in #11886
• ci: only create release if tag not exists by @everpcpc in #11909

Documentation 📔

• docs(blog): What's Fresh in Databend v1.1 by @PsiACE in #11074
• docs(weekly): add this week in databend 89 by @PsiACE in #11085
• docs: docker setup by @soyeric128 in #11100
• docs: moved metrics content to monitor by @soyeric128 in #11101
• docs: added youtube video by @soyeric128 in #11103
• docs: refactor quick start materials by @soyeric128 in #11114
• docs: refactor metrics by @soyeric128 in #11115
• docs: Update social media links by @soyeric128 in #11120
• docs: change restore table to flashback by @BohuTANG in #11121
• docs: make README more shrink by @BohuTANG in #11122
• docs: fix readme broken cosine function link by @kemingy in #11137
• docs: List stage files by @soyeric128 in #11139
• docs: refactor FILE UPLOAD API by @soyeric128 in #11157
• docs: remove external link and add system caches by @BohuTANG in #11161
• docs: display to_nullable return type by @wubx in #11167
• docs: to_nullable updates by @soyeric128 in #11170
• docs: add json_path_query by @BohuTANG in #11174
• docs: add cover for blog by @xudong963 in #11178
• docs: better functions by @BohuTANG in #11181
• docs: Update 02-jdbc.md by @wubx in #11186
• docs: Update 02-jdbc.md by @soyeric128 in #11195
• docs(blog): add Optimizing Compilation for Databend by @PsiACE in #11189
• docs(release): add v1.1 release note by @PsiACE in #11187
• docs: add this week in databend 90 by @PsiACE in #11197
• docs: New cloud page by @soyeric128 in #11198
• docs: sounds like by @soyeric128 in #11199
• docs: refine the copy by @BohuTANG in #11202
• docs: refine the AI functions by @BohuTANG in #11205
• docs: improve optimize table by @BohuTANG in #11207
• docs: make the sql reference related docs self-contained by @flaneur2020 in #11209
• docs: upload to external stage by @soyeric128 in #11213
• docs: load data from stage o bucket by @soyeric128 in #11221
• docs: update table default value by @BohuTANG in #11222
• docs: add drivers index page by @BohuTANG in #11223
• docs: update usage for bendsql by @everpcpc in #11229
• docs: add more example & instruction for bendsql by @everpcpc in #11230
• docs(develop): add rust by @PsiACE in #11236
• docs: renamed bendsql to BendSQL by @soyeric128 in #11240
• docs: updated flowcharts by @soyeric128 in #11237
• docs: added YouTube video by @soyeric128 in #11264
• docs: add link to data types by @BohuTANG in #11289
• docs(weekly): add this week in databend 91 by @PsiACE in #11292
• docs: Update 2023-04-30-databend-weekly-91.md by @soyeric128 in #11294
• docs: set_var by @soyeric128 in #11311
• docs: Load data by @soyeric128 in #11310
• docs(develop/python): add close connect by @PsiACE in #11316
• docs: why Optimize by @soyeric128 in #11329
• docs: data transform tutorials by @soyeric128 in #11323
• docs(weekly): add this week in databend 92 by @PsiACE in #11353
• docs: refactor docs about drivers by @hantmac in #11335
• docs: column alias in WHERE by @soyeric128 in #11383
• docs: Purge retention period by @soyeric128 in #11387
• docs: column name & alias conventions by @soyeric128 in #11389
• docs: combine stage, load, and unload by @soyeric128 in #11399
• docs: add feature licensing on license faq by @ZhiHanZ in #11405
• docs: display the license FAQ link in the README by @flaneur2020 in #11408
• docs: fix typo. by @youngsofun in #11426
• docs: data transform updates by @soyeric128 in #11417
• docs: add vacuum table doc by @lichuang in #11396
• docs(weekly): add this week in databend 93 by @PsiACE in #11446
• docs: utc timezone for logs by @soyeric128 in #11463
• docs: add link to ee for vacuum table feature by @BohuTANG in #11469
• docs: new date&time functions by @soyeric128 in #11465
• docs: VACUUM TABLE updates by @soyeric128 in #11481
• docs: transform data from uri by @soyeric128 in #11501
• docs: vacuum table updates by @soyeric128 in #11498
• docs: date&time specifier list by @soyeric128 in #11499
• docs: Move up EE page by @soyeric128 in #11513
• docs: add py connection parameters by @hantmac in #11512
• docs: fix "create external" example by @soyeric128 in #11523
• docs: added introduced version infor by @soyeric128 in #11515
• docs: rephrase by @soyeric128 in #11524
• docs:Update 03-jdbc.md by @wubx in #11526
• docs: doc templates by @soyeric128 in #11535
• docs(weekly): add this week in databend 94 by @PsiACE in #11534
• docs: contribute documents guide by @soyeric128 in #11547
• docs: filter files with FILES by @soyeric128 in #11548
• docs: update bendsql command by @ZhengLin-Li in #11563
• docs: Query metadata by @soyeric128 in #11566
• docs: add support for azure openai service by @BohuTANG in #11577
• docs: minor fixes by @soyeric128 in #11575
• docs: bitmap functions by @soyeric128 in #11574
• docs(weekly): add this week in databend 95 by @PsiACE in #11606
• docs: Lazy topn threshold by @soyeric128 in #11609
• docs: bitmap datatype by @soyeric128 in #11607
• docs: bitmap functions by @soyeric128 in #11615
• docs(website): try to add dead urls check by @PsiACE in #11594
• docs(website): fix yml lint by @PsiACE in #11629
• docs: tencent cos by @soyeric128 in #11628
• docs: deploy cluster by @soyeric128 in #11631
• docs: fix typos in 30-license-faqs.md by @tisonkun in #11638
• docs(blog): add tableau by @PsiACE in #11652
• docs(catalog): correct information_schema is supported by @tisonkun in #11658
• docs: load-unload-overview illustration by @soyeric128 in #11664
• docs(blog): add this week in databend 96 by @PsiACE in #11663
• docs: max_files option by @soyeric128 in #11670
• docs: Deploy mode updates by @soyeric128 in #11668
• docs: sql alias by @soyeric128 in #11673
• docs: docker deploy updates by @soyeric128 in #11676
• docs: column alias updates by @soyeric128 in #11680
• docs: set var updates by @soyeric128 in #11690
• docs: updated atomic tip by @soyeric128 in #11691
• docs: system.indexes by @soyeric128 in #11694
• docs: updated description for lazy_topn_threshold by @soyeric128 in #11695
• docs: use databend as vector sink by @soyeric128 in #11700
• docs(weekly): add this week in databend 97 by @PsiACE in #11722
• docs(JDBC): close statement after use. by @youngsofun in #11730
• docs: replace with stage attachments by @soyeric128 in #11725
• docs: computed columns by @soyeric128 in #11565
• docs: Update 2023-06-11-databend-weekly-97 by @soyeric128 in #11734
• docs: array_agg updates by @soyeric128 in #11735
• docs: Column position by @soyeric128 in #11744
• docs: table alias by @soyeric128 in #11753
• docs: max bitmap by @soyeric128 in #11754
• docs: add computed columns to EE by @soyeric128 in #11736
• docs: delete with subquery by @soyeric128 in #11769
• docs: date_format by @soyeric128 in #11774
• docs(weekly): add this week in databend 98 by @PsiACE in #11789
• docs: Update object-keys.md by @wubx in #11797
• docs: Update 2023-06-18-databend-weekly-98.md by @soyeric128 in #11810
• docs: select $N from csv by @soyeric128 in #11813
• docs: support ON_ERROR=abort_N by @youngsofun in #11822
• docs: rename column by @soyeric128 in #11821
• docs: on_error by @soyeric128 in #11825
• docs: max_threads hint for copy into by @soyeric128 in #11830
• docs: datediff by @soyeric128 in #11828
• docs: fix file_format for copy into by @soyeric128 in #11846
• docs: flink cdc by @soyeric128 in #11851
• docs(weekly): add this week in databend 99 by @PsiACE in #11862
• docs: datax by @soyeric128 in #11857
• docs: Addax by @soyeric128 in #11873
• docs: improved sidebar menu by @soyeric128 in #11881
• docs: stage files with presign by @soyeric128 in #11900

Others 📒

• chore: enable share statefull test and fix read share table bug by @lichuang in #11075
• chore(planner): Add benchmark utility for constraint by @leiysky in #11070
• chore: Bump to reqsign 0.9.2 for helping debug by @Xuanwo in #11107
• chore(ast): merge parsers for query and set_operation by @andylokandy in #11106
• chore: enable tpcds q64 by @xudong963 in #11118
• chore(ci): fix do not use bash dict in dev_setup by @everpcpc in #11147
• chore(ci): fix sccache path by @everpcpc in #11149
• chore(benchmark): only run tpch on Large by @everpcpc in #11150
• chore: use BlockBuilder in SerializeDataTransform by @zhyass in #11159
• chore(planner): remove BoundInternalColumnRef from ScalarExpr by @andylokandy in #11156
• chore: refine the default column value error by @BohuTANG in #11169
• chore: Bump reqsign to fix token not encoded by @Xuanwo in #11172
• chore: add openai http request timeout by @BohuTANG in #11191
• chore: reduce benchmark round to 3 by @xudong963 in #11194
• chore(function): simplify TryAdaptor by @andylokandy in #11196
• chore: more test cases of fuse compat by @dantengsky in #11214
• chore(deps): bump enumflags2 from 0.7.5 to 0.7.7 by @dependabot in #11217
• chore: refine take_compacted_indices by @Dousir9 in #11215
• chore: fix TPCDS Q39 flaky test by @Dousir9 in #11235
• chore(ci): upgrade bendsql to v0.3.4 [skip ci] by @everpcpc in #11249
• chore: fuse-compact test case, compaction of mixed versioned segments by @dantengsky in #11218
• chore: make is_nullable to be true for ArrowRs Field by @sundy-li in #11251
• chore(query): try fix build failure by @zhang2014 in #11259
• chore(ci): use larger runner for build aarch64 musl by @everpcpc in #11263
• chore(ci): check disk before & after download [skip ci] by @everpcpc in #11269
• chore: polish various license content by @tisonkun in #11278
• chore: contributor-friendly error message by @SkyFan2002 in #11277
• chore(build-tool): link musl with mold [skip ci] by @everpcpc in #11286
• chore(build-tool): fixes for shellcheck by @everpcpc in #11287
• chore: enable cast decimal 256 to 128 by @sundy-li in #11298
• chore: Add rust-analyzer in rust-toolchain by @Xuanwo in #11300
• chore: move tpch, tpcds tests to sqllogictest by @xudong963 in #11306
• chore: refactor snapshot_format_version function, fix possible snapshot version missmatch bug by @lichuang in #11328
• chore(ci): use larger JEMALLOC_SYS_WITH_LG_PAGE for aarch64 build by @everpcpc in #11334
• chore: remove broadcast join limitation by @xudong963 in #11109
• chore: add tpch, tpcds test to cluster by @xudong963 in #11358
• chore: improve max_of_max_precision by @Dousir9 in #11363
• chore: specify database in benchmark by @DongHaowen in #11362
• chore: rename worker_num to worker_id. by @youngsofun in #11374
• chore(query): call by name for prune cache key by @zhang2014 in #11382
• chore: correct the size of decimal ops by @sundy-li in #11385
• chore(query): z3-features2 by @sundy-li in #11400
• chore(meta-ctl): refine stderr output for metactl by @drmingdrmer in #11404
• chore: new release regex by @Mehrbod2002 in #11415
• chore(planner): allow untrustworthy column ref type of constant folding rule by @andylokandy in #11423
• chore: add kill query randomly vacuum table test by @lichuang in #11424
• chore: use lenient semver for protoc by @SkyFan2002 in #11422
• chore: http handler release context/session earlier. by @youngsofun in #11403
• chore: simple read segments by @zhyass in #11433
• chore(planner): postpone type unification for window frame by @andylokandy in #11429
• chore(planner): do not use data type in optimizer by @andylokandy in #11436
• chore: fix ee ci workflow by @ZhiHanZ in #11447
• chore: add more vacuum table test by @lichuang in #11462
• chore(planner): fix a bug in require_property by @Dousir9 in #11444
• chore: tune the error message on setting max_storage_io_requests by @flaneur2020 in #11471
• chore: Changed releases links by @Chasen-Zhang in #11483
• chore: reduce an extra plan_sql in http api by @sundy-li in #11485
• chore(query): make row fetcher lazy read segments by @sundy-li in #11508
• chore(query): skip TPCDS q35 by @zhang2014 in #11532
• chore: disable bushy join by @Dousir9 in #11549
• chore: warning the unknown session settings by @flaneur2020 in #11570
• chore(planner): Replace owned structures with Arc to reduce memory consumption by @leiysky in #11573
• chore: reduce state for hash join by @xudong963 in #11580
• chore: rename variable name by @gitccl in #11603
• chore: add large dataset test for iejoin by @xudong963 in #11608
• chore(expr): remove run_auto_type by @andylokandy in #11612
• chore(expr): add cast_scalar util function by @andylokandy in #11611
• chore: Bump opendal to v0.36 by @Xuanwo in #11624
• chore(planner): Minor refactor property derivation by @leiysky in #11618
• chore(planner): improve constant fold by @andylokandy in #11432
• chore: fix main build error by @xudong963 in #11630
• chore: refactor physical join by @xudong963 in #11622
• chore: remove take_string_types by @Dousir9 in #11645
• chore: add tpcds q64 by @Dousir9 in #11655
• chore: contributor-friendly error message by @JackTan25 in #11660
• chore: Rewrite IGNORE_TEXT by @Chasen-Zhang in #11667
• chore: disable non_equi_conditions in eager aggregation by @Dousir9 in #11679
• chore(storage): adjust the file dirs by @zhyass in #11687
• chore(deps): bump xml-rs from 0.8.4 to 0.8.14 by @dependabot in #11674
• chore(expr): check type of BlockEntry by @andylokandy in #11684
• chore(planner): shrink constant type by @andylokandy in #11688
• chore(storage): remove compact in recluster by @zhyass in #11697
• chore(planner): remove RuleFoldConstant by @andylokandy in #11702
• chore(executor): Add a switch to enable profiling in cluster mode by @leiysky in #11731
• chore: fix take_compact by @Dousir9 in #11737
• chore: support lazy materialization for more queries. by @RinChanNOWWW in #11738
• chore(expr): add FunctionCtx in calc_domain by @andylokandy in #11781
• chore: fix join by @Dousir9 in #11788
• chore: state combinator wraps the whole function. by @RinChanNOWWW in #11795
• chore: improve left join by @Dousir9 in #11793
• chore: change broadcast join threshold by @xudong963 in #11812
• chore: rename column check referenced by computed columns by @b41sh in #11805
• chore: remove disk cache when query init by @lichuang in #11827
• chore(build): bump rust version to nightly-2023-06-09 by @leiysky in #11803
• chore: should not regist session in non-background service by @ZhiHanZ in #11854
• chore: add integration tests to avoid session leak on start by @ZhiHanZ in #11855
• chore: fast path for hash join if build side is empty by @xudong963 in #11835
• chore: fast path for left-related join if build side is empty by @xudong963 in #11865
• chore(planner): improve table schema of system.query_profile by @leiysky in #11866
• chore: add sqllogic tests for aggregating index. by @RinChanNOWWW in #11860
• chore(planner): add more execution information in system.query_profile by @leiysky in #11876
• chore(query): fix typo by @zhang2014 in #11893
• chore: add unit tests and sql logic tests for union query. by @RinChanNOWWW in #11883
• chore: fix adjust_ndv by @Dousir9 in #11890
• chore(query): query entry display ${USER} and ${PASSWORD} by @b41sh in #11895
• chore(query): bump arrow by @sundy-li in #11896
• chore(ci): add note for pr docker image by @everpcpc in #11901
• chore: support adding default root user for docker image by @b41sh in #11902
• chore(python): fix python release action by @sundy-li in #11906
• chore(meta): simplify testing codes by @drmingdrmer in #11907
• chore(query): rename benefit_column_prune to support_column_projection by @sundy-li in #11908

New Contributors

• @neil4dong made their first contribution in #11043
• @JackTan25 made their first contribution in #11290
• @Mehrbod2002 made their first contribution in #11367
• @DongHaowen made their first contribution in #11362
• @silver-ymz made their first contribution in #11487
• @Jake-00 made their first contribution in #11503
• @gitccl made their first contribution in #11507
• @ZhengLin-Li made their first contribution in #11563
• @jonahgao made their first contribution in #11718
• @akoshchiy made their first contribution in #11783

Full Changelog: v1.1.0-nightly...v1.2.0-nightly