-
Notifications
You must be signed in to change notification settings - Fork 760
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs(weekly): add this week in databend 97 (#11722)
Signed-off-by: Chojan Shang <psiace@apache.org>
- Loading branch information
Showing
1 changed file
with
139 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,139 @@ | ||
--- | ||
title: "This Week in Databend #97" | ||
date: 2023-06-11 | ||
slug: 2023-06-11-databend-weekly | ||
tags: [databend, weekly] | ||
description: "Stay up to date with the latest weekly developments on Databend!" | ||
contributors: | ||
- name: andylokandy | ||
- name: b41sh | ||
- name: Chasen-Zhang | ||
- name: dantengsky | ||
- name: dependabot[bot] | ||
- name: Dousir9 | ||
- name: drmingdrmer | ||
- name: JackTan25 | ||
- name: PsiACE | ||
- name: RinChanNOWWW | ||
- name: soyeric128 | ||
- name: sundy-li | ||
- name: tisonkun | ||
- name: Xuanwo | ||
- name: youngsofun | ||
- name: zhang2014 | ||
- name: zhyass | ||
authors: | ||
- name: PsiACE | ||
url: https://github.com/psiace | ||
image_url: https://github.com/psiace.png | ||
--- | ||
|
||
[Databend](https://github.com/datafuselabs/databend) is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: <https://app.databend.com> . | ||
|
||
## What's On In Databend | ||
|
||
Stay connected with the latest news about Databend. | ||
|
||
### Column Position | ||
|
||
Databend now supports using syntax like `$N` to indicate column positions, where `$2` means matching the second column. Furthermore, Databend also supports combining column position and column name in SQL statements. Here is a simple example: | ||
|
||
```SQL | ||
CREATE TABLE IF NOT EXISTS t1(a int, b varchar); | ||
INSERT INTO t1 values (1, 'a'), (2, 'b'); | ||
select $1, $2, a, b from t1; | ||
|
||
┌─────────────────────────────────┐ | ||
│ $1 │ $2 │ a │ b │ | ||
│ Int32 │ String │ Int32 │ String │ | ||
├───────┼────────┼───────┼────────┤ | ||
│ 1 │ a │ 1 │ a │ | ||
│ 2 │ b │ 2 │ b │ | ||
└─────────────────────────────────┘ | ||
``` | ||
|
||
We have also added column position support for `SELECT FROM` with NDJSON, and support for other formats is in progress. | ||
|
||
```sql | ||
select $1 from @my_stage | ||
|
||
copy into my_table from (select $1 from @my_stage t) | ||
``` | ||
|
||
Please note that when using `select` for NDJson, only `$1` is allowed. This stands for the entire line and has a variant type. On the other hand, when copying into `my_table` from `@my_stage`, matching is done by name of top-level fields. | ||
|
||
If you are interested in learning more, please check out the resources listed below: | ||
|
||
- [Issue | Feature: support $<col_position>](https://github.com/datafuselabs/databend/issues/11585) | ||
- [Issue | Feature: copy/select from stage by pos](https://github.com/datafuselabs/databend/issues/11581) | ||
- [PR | feat: support column position like $N](https://github.com/datafuselabs/databend/pull/11672) | ||
- [PR | feat: select from stage support NDJson](https://github.com/datafuselabs/databend/pull/11701) | ||
|
||
## Code Corner | ||
|
||
Discover some fascinating code snippets or projects that showcase our work or learning journey. | ||
|
||
### Learn Databend Workflows - Typos Check | ||
|
||
Databend now has a very complex workflow for handling code auditing, testing, benchmarking and release. Typos Check is undoubtedly the simplest part of it. Let's take a look at some of its contents together. | ||
|
||
Like other workflows, we need to use actions/checkout to check out the code. | ||
|
||
```yaml | ||
- uses: actions/checkout@v3 | ||
with: | ||
clean: "true" | ||
``` | ||
`typos-cli` is a source code spell checker that finds and corrects spelling mistakes in source code. It is fast enough to run on monorepos and has low false positives, making it suitable for use on PRs. | ||
|
||
```yaml | ||
- uses: baptiste0928/cargo-install@v1 | ||
with: | ||
crate: typos-cli | ||
args: --locked | ||
cache-key: typos-check | ||
``` | ||
|
||
We use `baptiste0928/cargo-install` to install dependencies. It is essentially the same as using `cargo install` in your GitHub workflows. Additionally, it allows for automatic caching of resulting binaries to speed up subsequent builds. | ||
|
||
```yaml | ||
- name: do typos check with typos-cli | ||
run: typos | ||
``` | ||
|
||
One thing to note is that `typos-cli` is the name of the crate, but the corresponding executable binary name is `typos`. | ||
|
||
If you are interested in learning more, please check out the resources listed below: | ||
|
||
- [Workflows | typos.yml](https://github.com/datafuselabs/databend/blob/main/.github/workflows/typos.yml) | ||
|
||
## Highlights | ||
|
||
We have also made these improvements to Databend that we hope you will find helpful: | ||
|
||
- Added support for distributed Top-N. | ||
- The default `lazy_topn_threshold` setting has been enabled to 1000. | ||
- To ensure security, the root user has been allowed to change the password. | ||
- Read *[Blog | Databend X Tableau](https://databend.rs/blog/2023-06-01-tableau)* to learn how to connect Databend for BI data analysis in Tableau. | ||
- Read *[Docs | Integrating Databend as a Sink for Vector](https://databend.rs/doc/integrations/data-tool/vector)* and *[Docs | Analyzing Nginx Access Logs with Databend](https://databend.rs/doc/use-cases/analyze-nginx-logs-with-databend-and-vector)* to understand how to integrate Vector and Databend. | ||
|
||
## What's Up Next | ||
|
||
We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend. | ||
|
||
### Add a Deduplication Label Field to the Rest API | ||
|
||
To ensure that data ingestion is idempotent, Databend already supports deduplication of DML through the use of a deduplication label. You can find more information on this feature at [Docs | Setting Commands - SET_VAR](https://databend.rs/doc/sql-commands/setting-cmds/set-var) . | ||
|
||
To facilitate cross-language driver integration, we could add a REST API field for the label. | ||
|
||
[Issue #11710 | Feature: support to bring deduplication label on stage attachment api](https://github.com/datafuselabs/databend/issues/11710) | ||
|
||
Please let us know if you're interested in contributing to this issue, or pick up a good first issue at <https://link.databend.rs/i-m-feeling-lucky> to get started. | ||
|
||
## Changelog | ||
|
||
You can check the changelog of Databend Nightly for details about our latest developments. | ||
|
||
**Full Changelog**: <https://github.com/datafuselabs/databend/compare/v1.1.55-nightly...v1.1.56-draft2> |
25c270b
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Successfully deployed to the following URLs:
databend – ./
databend.rs
databend.vercel.app
databend-git-main-databend.vercel.app
databend-databend.vercel.app