Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: correct column order #340

Merged
merged 20 commits into from
Nov 18, 2022
Merged

feat: correct column order #340

merged 20 commits into from
Nov 18, 2022

Conversation

dust1
Copy link
Contributor

@dust1 dust1 commented Oct 24, 2022

Which issue does this PR close?

Closes #187

Rationale for this change

Table' columns order is not what user define when create table

What changes are included in this PR?

I modified the 'add_key_column' and 'add_normal_column' methods in 'schema.builder' and added 'primary_key_index' to record additional field primary key information.

Of course this affects the subsequent storage logic.

I removed num_key_size from RecordSchemaWithKey and added key_index to confirm the location of the primary key.

The end result:

mysql> CREATE TABLE `demo` (`name` string TAG, `value` double NOT NULL, `t` timestamp NOT NULL, TIMESTAMP KEY(t)) ENGINE=Analytic with (enable_ttl='false');
Query OK, 0 rows affected (0.02 sec)

mysql> insert into demo (name, value, t) values ('zaaaaasss', 1.112, 1651737061500);
Query OK, 1 row affected (0.00 sec)

mysql> insert into demo (name, value, t) values ('zaaaaasss', 1.112, 1651737061100);
Query OK, 1 row affected (0.00 sec)

mysql> select * from demo;
+-----------+-------+---------------+---------------------+
| name      | value | t             | tsid                |
+-----------+-------+---------------+---------------------+
| zaaaaasss | 1.112 | 1651737061100 | 3511143416174120951 |
| zaaaaasss | 1.112 | 1651737061500 | 3511143416174120951 |
+-----------+-------+---------------+---------------------+
2 rows in set (0.00 sec)

mysql> desc demo;
+-------+-----------+------------+-------------+--------+
| name  | type      | is_primary | is_nullable | is_tag |
+-------+-----------+------------+-------------+--------+
| name  | string    |          0 |           1 |      1 |
| value | double    |          0 |           0 |      0 |
| t     | timestamp |          1 |           0 |      0 |
| tsid  | uint64    |          1 |           0 |      0 |
+-------+-----------+------------+-------------+--------+
4 rows in set (0.00 sec)

Are there any user-facing changes?

How does this change test

I change two unit test in schema.rs. in L1597 and L1600, I changed the field validation order, make the order of the fields consistent with the order in which they were created

@jiacai2050
Copy link
Contributor

@dust1 It seems your branch is kinds of stale, please rebase with latest main branch.

@dust1
Copy link
Contributor Author

dust1 commented Oct 26, 2022

ok

@ShiKaiWi
Copy link
Member

In my understanding, these changes are sufficient, but when I test the new table, I find that the primary_key_index is empty in the data re-read by wal log. I need more time to check the reason

@dust1 Have you figured out this problem? I can help do some troubleshooting if you will. Don't worry about the time because this pr is not urgent for us.

@dust1 dust1 marked this pull request as ready for review November 10, 2022 13:33
@dust1
Copy link
Contributor Author

dust1 commented Nov 10, 2022

In my understanding, these changes are sufficient, but when I test the new table, I find that the primary_key_index is empty in the data re-read by wal log. I need more time to check the reason

@dust1 Have you figured out this problem? I can help do some troubleshooting if you will. Don't worry about the time because this pr is not urgent for us.

thank you @ShiKaiWi 🥰. Maybe I've already done this pr

@jiacai2050 jiacai2050 self-requested a review November 11, 2022 03:24
common_types/src/schema.rs Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
common_types/src/schema.rs Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
proto/protos/common.proto Outdated Show resolved Hide resolved
sql/src/planner.rs Show resolved Hide resolved
sql/src/planner.rs Outdated Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
sql/src/planner.rs Outdated Show resolved Hide resolved
@jiacai2050 jiacai2050 changed the title edit: column order use user define feat: column order use user define Nov 17, 2022
@jiacai2050 jiacai2050 changed the title feat: column order use user define feat: correct column order Nov 17, 2022
sql/src/planner.rs Show resolved Hide resolved
sql/src/planner.rs Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
common_types/src/schema.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@jiacai2050 jiacai2050 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiacai2050 jiacai2050 merged commit 6c0bca1 into apache:main Nov 18, 2022
@dust1 dust1 deleted the issue187 branch November 18, 2022 02:51
chunshao90 pushed a commit to chunshao90/ceresdb that referenced this pull request May 15, 2023
* edit: column order use user define

* edit: update commit

* fix: some error

* fix: key comparableInnternalKey encode

* edit: select

* fix: insert data compare use primary key

* fix: unit test

* edit: remote annotation, fix large enum variant

* fix

* fix: unit test

* fix: unit test

* fix: project_record_schema_with_key

* fix: unit test

* edit: more simple code

* edit: modification of description
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Column order is not what user define when create table
3 participants