-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dramatic performance degradation between JSqlParser 4.0 and 4.2 #1397
Comments
@manticore-projects We have to check those complex expression extension for case when constructs. I have a performance degradation of 10x and more here. At least I suspect those changes to be the problem. |
Can you give me the particular SQL statements please, which you have used for the test and maybe point out how exactly you measured? Should we involve PMD as Test Dependency to measure impact of changes more reliably? Also, we would need to discuss the objectives "correctness" vs. "performance". In my own opinion, a parser must be as correct as possible at first, and only after that should be as fast as possible. I was reluctant to sacrifice correctness over performance. But of course we should look into optimizations, when a particular construct causes such performance deterioration. |
Sure, but a degradation of 10x is huge. I am working on an example SQL. For statistics of this I wrote https://github.com/JSQLParser/javacc-perf-diag. |
I was not aware, that there is another Performance related Test Suite aside our normal integration tests (with defined 2sec timeouts). So far I was satisfied when the normal tests ran through, within 30 seconds in total and without a time out exception for a single test. (Just explaining where I come from.) Of course I share your concern about the performance deterioration. In my opinion, it's a two step:
I believe investing into step 1 makes sense because it will be a re-occurring problem: Correctness unfortunately competes with performance and every single improvement on the Correctness will jeopardize the performance. So we would need to run this benchmarking tool before we commit anything and make it part of the CI process. It also means, that we should find a smart way to extract all SQL statements from the Test Suite. Currently they are burried inside the Java Code and not easy to access from outside. |
any plans to fix this issue? I meet the same questions too. |
With what query and which version on JSQLParser exactly please. |
@chensk0601 The investigation is in progress ... I am right now collecting SQLs that have performance issues for version 4.2 and had no performance issues in version v4 and below. If you have some, feel free to add those at this issue. |
Here is one SQL. The problem is, that parsing this using JSqlParser 4.2 uses nearly 10 times of token reads, lookaheads than version 4.0. Sure there could be a smaller example, but this is the start. |
Simplify the Primary Expression Production Try to simple parse without Complex Expressions first, before parsing complex and slow (if supported by max nesting depth) Add Test cases for issues JSQLParser#1397 and JSQLParser#1438 Update Libraries to its latest version Remove JUnit 4 from Gradle
* Adjust Gradle to JUnit 5 Parallel Test execution Gradle Caching Explicitly request for latest JavaCC 7.0.10 * Do not mark SpeedTest for concurrent execution * Remove unused imports * Adjust Gradle to JUnit 5 Parallel Test execution Gradle Caching Explicitly request for latest JavaCC 7.0.10 * Do not mark SpeedTest for concurrent execution * Remove unused imports * Performance Improvements Simplify the Primary Expression Production Try to simple parse without Complex Expressions first, before parsing complex and slow (if supported by max nesting depth) Add Test cases for issues #1397 and #1438 Update Libraries to its latest version Remove JUnit 4 from Gradle * Appease PMD * Update Gradle Plugins to its latest versions Let Parser timeout after 6 seconds and fail gently Add a special test verifying the clean up after timeout * Revert unintended changes to the Test Resources * Appease PMD/Codacy * Correct the Gradle "+" dependencies * Bump version to 4.4.-SNAPSHOT * update build file * revert unwarranted changes in test files * strip the Exception Class Name from the Message * maxDepth = 10 collides with the Parser Timeout = 6 seconds * License Headers Unused imports * Bump version to 4.5-SNAPSHOT Reduce test loops to fit intothe timeout
这是来自QQ邮箱的假期自动回复邮件。
将在最快的时间回复
|
Some of our included PRs had a dramatic performance impact. So they need to be revisited. This log comes from the same SQL which uses non recursive large case when statements.
JSqlParser 4.2
JSqlParser 4.0
If you look at the methods call log, some methods are called 10 times and more of the version 4.0.
Maybe we have to rewind some PRs for this.
The text was updated successfully, but these errors were encountered: