Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add compat test for fuse engine #6557

Closed
BohuTANG opened this issue Jul 9, 2022 · 5 comments · Fixed by #7018
Closed

Add compat test for fuse engine #6557

BohuTANG opened this issue Jul 9, 2022 · 5 comments · Fixed by #7018
Assignees
@drmingdrmer
Labels
C-improvement Category: improvement

Comments

@BohuTANG
Copy link
Member

BohuTANG commented Jul 9, 2022 •
edited
Loading

Summary
Like:
https://github.com/datafuselabs/databend/blob/main/.github/actions/test_compat/action.yml

Ensure that each of our versions is compatible with the previous ones.
@BohuTANG BohuTANG added the C-improvement Category: improvement label Jul 9, 2022
@flaneur2020 flaneur2020 mentioned this issue Jul 11, 2022
Closed
@BohuTANG BohuTANG mentioned this issue Jul 19, 2022
Merged
@BohuTANG
Copy link
Member Author

Is it possible to add a compact test for databend-query with different versions(current and previous version) to https://github.com/datafuselabs/databend/blob/main/tests/compat/test-compat.sh?

@drmingdrmer @everpcpc

@drmingdrmer
Copy link
Member

Is it possible to add a compact test for databend-query with different versions(current and previous version) to https://github.com/datafuselabs/databend/blob/main/tests/compat/test-compat.sh?

Suppose the query is only compatible with a limited number of history versions. In that case, the query needs two versions defined: VER is the version of the query executable, and MIN_COMPATIBLE is the oldest version this executable is compatible with.

If the query must be compatible with all history versions, MIN_COMPATIBLE is unnecessary.

The fuse-data-compat-test would be:

  • Download an old query executable, bring it up, and write some data into it.
  • Shut it down, bring up the latest executable, and run some read-only tests.

And the fuse-data-Compatability may differ slightly from the meta service compatibility.
There may be 3 versions of fuse-data: A, B, C such that B can read A, C can read A but can not read B. In other words, the fuse-data format may not evolve incrementally. Thus maybe fuse data compatibility tests have to test every possible version combinations.

@BohuTANG
Copy link
Member Author

BohuTANG commented Aug 2, 2022

@drmingdrmer will start this task with meta-srv compact framework. cc @dantengsky

@drmingdrmer drmingdrmer self-assigned this Aug 2, 2022
@drmingdrmer
Copy link
Member

I'm trying to build up a script that will:

    1. Bring up databend-query of the specified version, and run a bunch of SQL to feed data;
    1. Shut it down and bring up the latest databend-query, and run another bunch of SQL to load the data.

My question is:
What SQL statements will be enough to cover the compatibility test in these 2 phases?

drmingdrmer added a commit to drmingdrmer/databend that referenced this issue Aug 5, 2022
@drmingdrmer
ci(query): test fuse-table compatibility
854b525 
Fuse table compatability test

This script tests whether a newer version databend-query can read fuse table data written
by a older version databend-query.

Usage

```shell
tests/fuse-compat/test-fuse-compat.sh <old_ver>
```

E.g. `tests/fuse-compat/test-fuse-compat.sh 0.7.151` tests if the fuse-table written
by **databend-query-0.7.151** can be read by **current** version databend-qeury.

Prerequisites

- Current version of databend-query and databend-meta must reside in `./bins`:
    - `./bins/current/databend-query`
    - `./bins/current/databend-meta`

    Since building a binary takes several minutes,
    this step is usually done by the calling process, e.g., the CI script.

Testing data

- Suite `tests/fuse-compat/compat-logictest/fuse_compat_write` writes data into a fuse table via an old version query.
- Suite `tests/fuse-compat/compat-logictest/fuse_compat_read` reads the data via current version query.

Fuse table maintainers update these two `logictest` scripts to let the write/read
operations cover fuse-table features.

- Part of databendlabs#6557

TODO

- [ ] Complete logictest script `tests/fuse-compat/compat-logictest/{fuse_compat_write|fuse_compat_read}` to cover fuse-table features.
This was referenced Aug 5, 2022
Merged
Merged
@drmingdrmer
Copy link
Member

The version to assert being compatible with the latest databend-query has to be explicitly specified:

https://github.com/datafuselabs/databend/blob/a65c8d33269008ac775f2232260d96e0750b979c/.github/actions/fuse_compat/action.yml#L29-L30

BohuTANG pushed a commit that referenced this issue Aug 7, 2022
@drmingdrmer
ci(fuse-table): add data types test to fuse-compat test
09968e1 
- Fix: #6557
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@drmingdrmer drmingdrmer

Labels
C-improvement Category: improvement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ci(fuse-table): add data types test to fuse-compat test drmingdrmer/databend
3 participants
@drmingdrmer @BohuTANG @dantengsky