Skip to content

Commit

Permalink
docs(weekly): add this week in databend 97 (#11722)
Browse files Browse the repository at this point in the history
Signed-off-by: Chojan Shang <psiace@apache.org>
  • Loading branch information
PsiACE authored Jun 12, 2023
1 parent 7496ef4 commit 25c270b
Showing 1 changed file with 139 additions and 0 deletions.
139 changes: 139 additions & 0 deletions website/blog/2023-06-11-databend-weekly-97.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
---
title: "This Week in Databend #97"
date: 2023-06-11
slug: 2023-06-11-databend-weekly
tags: [databend, weekly]
description: "Stay up to date with the latest weekly developments on Databend!"
contributors:
- name: andylokandy
- name: b41sh
- name: Chasen-Zhang
- name: dantengsky
- name: dependabot[bot]
- name: Dousir9
- name: drmingdrmer
- name: JackTan25
- name: PsiACE
- name: RinChanNOWWW
- name: soyeric128
- name: sundy-li
- name: tisonkun
- name: Xuanwo
- name: youngsofun
- name: zhang2014
- name: zhyass
authors:
- name: PsiACE
url: https://github.com/psiace
image_url: https://github.com/psiace.png
---

[Databend](https://github.com/datafuselabs/databend) is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: <https://app.databend.com> .

## What's On In Databend

Stay connected with the latest news about Databend.

### Column Position

Databend now supports using syntax like `$N` to indicate column positions, where `$2` means matching the second column. Furthermore, Databend also supports combining column position and column name in SQL statements. Here is a simple example:

```SQL
CREATE TABLE IF NOT EXISTS t1(a int, b varchar);
INSERT INTO t1 values (1, 'a'), (2, 'b');
select $1, $2, a, b from t1;

┌─────────────────────────────────┐
│ $1 │ $2 │ a │ b │
│ Int32 │ String │ Int32 │ String │
├───────┼────────┼───────┼────────┤
1 │ a │ 1 │ a │
2 │ b │ 2 │ b │
└─────────────────────────────────┘
```

We have also added column position support for `SELECT FROM` with NDJSON, and support for other formats is in progress.

```sql
select $1 from @my_stage

copy into my_table from (select $1 from @my_stage t)
```

Please note that when using `select` for NDJson, only `$1` is allowed. This stands for the entire line and has a variant type. On the other hand, when copying into `my_table` from `@my_stage`, matching is done by name of top-level fields.

If you are interested in learning more, please check out the resources listed below:

- [Issue | Feature: support $<col_position>](https://github.com/datafuselabs/databend/issues/11585)
- [Issue | Feature: copy/select from stage by pos](https://github.com/datafuselabs/databend/issues/11581)
- [PR | feat: support column position like $N](https://github.com/datafuselabs/databend/pull/11672)
- [PR | feat: select from stage support NDJson](https://github.com/datafuselabs/databend/pull/11701)

## Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

### Learn Databend Workflows - Typos Check

Databend now has a very complex workflow for handling code auditing, testing, benchmarking and release. Typos Check is undoubtedly the simplest part of it. Let's take a look at some of its contents together.

Like other workflows, we need to use actions/checkout to check out the code.

```yaml
- uses: actions/checkout@v3
with:
clean: "true"
```
`typos-cli` is a source code spell checker that finds and corrects spelling mistakes in source code. It is fast enough to run on monorepos and has low false positives, making it suitable for use on PRs.

```yaml
- uses: baptiste0928/cargo-install@v1
with:
crate: typos-cli
args: --locked
cache-key: typos-check
```

We use `baptiste0928/cargo-install` to install dependencies. It is essentially the same as using `cargo install` in your GitHub workflows. Additionally, it allows for automatic caching of resulting binaries to speed up subsequent builds.

```yaml
- name: do typos check with typos-cli
run: typos
```

One thing to note is that `typos-cli` is the name of the crate, but the corresponding executable binary name is `typos`.

If you are interested in learning more, please check out the resources listed below:

- [Workflows | typos.yml](https://github.com/datafuselabs/databend/blob/main/.github/workflows/typos.yml)

## Highlights

We have also made these improvements to Databend that we hope you will find helpful:

- Added support for distributed Top-N.
- The default `lazy_topn_threshold` setting has been enabled to 1000.
- To ensure security, the root user has been allowed to change the password.
- Read *[Blog | Databend X Tableau](https://databend.rs/blog/2023-06-01-tableau)* to learn how to connect Databend for BI data analysis in Tableau.
- Read *[Docs | Integrating Databend as a Sink for Vector](https://databend.rs/doc/integrations/data-tool/vector)* and *[Docs | Analyzing Nginx Access Logs with Databend](https://databend.rs/doc/use-cases/analyze-nginx-logs-with-databend-and-vector)* to understand how to integrate Vector and Databend.

## What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

### Add a Deduplication Label Field to the Rest API

To ensure that data ingestion is idempotent, Databend already supports deduplication of DML through the use of a deduplication label. You can find more information on this feature at [Docs | Setting Commands - SET_VAR](https://databend.rs/doc/sql-commands/setting-cmds/set-var) .

To facilitate cross-language driver integration, we could add a REST API field for the label.

[Issue #11710 | Feature: support to bring deduplication label on stage attachment api](https://github.com/datafuselabs/databend/issues/11710)

Please let us know if you're interested in contributing to this issue, or pick up a good first issue at <https://link.databend.rs/i-m-feeling-lucky> to get started.

## Changelog

You can check the changelog of Databend Nightly for details about our latest developments.

**Full Changelog**: <https://github.com/datafuselabs/databend/compare/v1.1.55-nightly...v1.1.56-draft2>

1 comment on commit 25c270b

@vercel
Copy link

@vercel vercel bot commented on 25c270b Jun 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Successfully deployed to the following URLs:

databend – ./

databend.rs
databend.vercel.app
databend-git-main-databend.vercel.app
databend-databend.vercel.app

Please sign in to comment.