LowCardinality support?

[sundy-li]

Hi, will this awesome driver support LowCardinality ?

[kshvakov]

Hi, no yet.

[agaurav]

@kshvakov It seems most of the bugs with LowCardinality have been fixed and the experimental feature label has also been removed. I will be great to have support for it in the most popular ch driver.

Have you started working on this ?
I will very much like to help out, in case you haven't.

[kshvakov]

@agaurav Yes, at the moment LowCardinality is production ready feature, I think I'll add this feature within a couple of weeks.

[agaurav]

@kshvakov It seems to be working fine for both inserts and select queries with LowCardinality(String) data type.
I don't know the internals of the drivers so might be missing a case which needs to be taken care of.

sample code to test insert and select.

package main

import (
	"github.com/jmoiron/sqlx"
	_ "github.com/kshvakov/clickhouse"
	"log"
	"time"
)

func main() {
	connect, err := sqlx.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
	if err != nil {
		log.Fatal(err)
	}

        _, err = connect.Exec(`
                    CREATE TABLE IF NOT EXISTS test.low_cardinality (
                           d Date,
                           x UInt32,
                           s LowCardinality(String)
                      ) ENGINE = MergeTree
                      ORDER BY (d,x,s)
                 `)

	var (
		tx, errtx   = connect.Begin()
		stmt, errst = tx.Prepare("INSERT INTO test.low_cardinality (d,x,s) VALUES (?, ?, ?)")
	)

	if errtx != nil {
		log.Fatal(errtx)
	}

	if errst != nil {
		log.Fatal(errst)
	}

	defer stmt.Close()

	for i := 0; i < 100; i++ {
		if _, err := stmt.Exec(
			time.Now(),
			i,
			"s",
		); err != nil {
			log.Fatal(err)
		}
	}

	if err := tx.Commit(); err != nil {
		log.Fatal(err)
	}

	rows, err := connect.Query("SELECT d,x,s FROM test.low_cardinality")
	if err != nil {
		log.Fatal(err)
	}
	defer rows.Close()

	for rows.Next() {
		var (
			d time.Time
			x uint32
			s string
		)
		if err := rows.Scan(&d, &x, &s); err != nil {
			log.Fatal(err)
		}
		log.Println(d, x, s)
	}
}

[kshvakov]

@agaurav It's a compatibility mode, ClickHouse server convert LowCardinality types on its side (see ClickHouse/ClickHouse#3879). It's no good for read/write performance.

[negbie]

+1

[John-belt]

Hi this seems like a wonderful feature to have on the driver. Is it available yet??

[John-belt]

@agaurav It's a compatibility mode, ClickHouse server convert LowCardinality types on its side (see ClickHouse/ClickHouse#3879). It's no good for read/write performance.

@kshvakov So does it work atleast as good as before without low cardinality?? We were thinking on adding lowCardinality to our schema, so if it is giving at aleast same performance for read/write as before, we can add it to the schema and wait till this is fully supported by the driver.

[sundy-li]

I think this issue could be closed. The compatibility mode is by default.
We are using LowCardinality with clickhouse-go on the production for 2 years, it works well.