Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support sql_calc_found_rows with limit in sharded keyspace #6680

Merged
merged 17 commits into from
Sep 14, 2020

Conversation

harshit-gangal
Copy link
Member

@harshit-gangal harshit-gangal commented Sep 9, 2020

Support queries using SQL_CALC_FOUND_ROWS. This is done by rewriting the queries according to https://dev.mysql.com/worklog/task/?id=12615:

Frequently, it would be cheaper to run the query with LIMIT and then a separate SELECT COUNT() for the same query, since COUNT() can make use of optimizations that can't be done when searching for the entire result set (e.g. filesort can be skipped for COUNT(*), whereas with CALC_FOUND_ROWS, we must disable some filesort optimizations to guarantee the right result). If you search online, you'll frequently find the advice to run two separate queries instead of using SQL_CALC_FOUND_ROWS, precisely for performance reasons.

So, vtgate will plan these queries by first solving the original query, without the SQL_CALC_FOUND_ROWS option, and then following up with a COUNT(*) query.

SELECT SQL_CALC_FOUND_ROWS col1, col2 FROM user WHERE col3 > "foo" LIMIT 10

will get be rewritten into two queries:

SELECT col1, col2 FROM user WHERE col3 > "foo" LIMIT 10
SELECT count(*) FROM user WHERE col3 > "foo" 

The first query will be used to return the results to the user, and the second will be used to save the found_rows state on the session in vtgate.

systay and others added 5 commits September 8, 2020 16:55
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
harshit-gangal and others added 6 commits September 9, 2020 20:46
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>

Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
@systay systay marked this pull request as ready for review September 10, 2020 09:08
@systay systay requested a review from sougou as a code owner September 10, 2020 09:08
@systay systay requested a review from deepthi September 10, 2020 09:09
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
go/vt/vtgate/engine/sql_calc_found_rows.go Outdated Show resolved Hide resolved
go/vt/vtgate/planbuilder/sql_calc_found_rows.go Outdated Show resolved Hide resolved
@@ -279,4 +279,3 @@
]
}
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reset this file change.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... Weird. On my machine, with 723fef02e checked out, that file has an extra line at the end, and git does not see any changes. Not sure what's going on...

Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
@askdba askdba added this to the v8.0 milestone Oct 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants