-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Current codebase does not handle 0 source rows well #20
Comments
I found a workaround. Modify the config file to allow a greater number of open connections: # Maximum number of connections to the MySQL allowed for this application.
- max_open_connections: 10
+ max_open_connections: 15 The max open connections value got up to 12 before the application completed. Once I inserted the test data, the open connections dropped to 1. This is testing against a local MariaDB 10.0.x instance, so the behavior is likely different against a remote database server, or even a newer version of MariaDB. |
atc0005
added a commit
that referenced
this issue
Sep 27, 2020
Earlier work was done on the `initial-prototype` branch and has only been slightly cleaned up for this release. Documentation, CI (GitHub Actions), dependency monitoring (Dependabot) and a host of other TODO items have not been completed as of this specific commit. Additionally, confirmed bugs such as GH-20 remain and should be fixed before the first v0.1.0 release. Some test files from the WhyAskWhy/mysql2sqlite-dev project are included in this commit. I am authorized to include them as well as use the project name. This commit provides two (mostly) working binaries: - `mysql2sqlite` - main binary to sync a MySQL database to a SQLite database - `check_mysql2sqlite` - Nagios plugin to validate source MySQL database against generated SQLite database As already noted, much, much work remains to polish and refactor this code. Even so, I am already using it in limited production use in order to help shape development efforts. Already this has proven useful in abandoning the plan to use Write-Ahead Logging (`WAL`) mode when interacting with the SQL database and instead use classic `DELETE` mode instead (compatibility). refs GH-19
atc0005
added a commit
that referenced
this issue
Sep 27, 2020
Earlier work was done on the `initial-prototype` branch and has only been slightly cleaned up for this release. Documentation, CI (GitHub Actions), dependency monitoring (Dependabot) and a host of other TODO items have not been completed as of this specific commit. Additionally, confirmed bugs such as GH-20 remain and should be fixed before the first v0.1.0 release. Some test files from the WhyAskWhy/mysql2sqlite-dev project are included in this commit. I am authorized to include them as well as use the project name. This commit provides two (mostly) working binaries: - `mysql2sqlite` - main binary to sync a MySQL database to a SQLite database - `check_mysql2sqlite` - Nagios plugin to validate source MySQL database against generated SQLite database As already noted, much, much work remains to polish and refactor this code. Even so, I am already using it in limited production use in order to help shape development efforts. Already this has proven useful in abandoning the plan to use Write-Ahead Logging (`WAL`) mode when interacting with the SQL database and instead use classic `DELETE` mode instead (compatibility). refs GH-19
atc0005
added a commit
that referenced
this issue
Sep 29, 2020
For both binaries, check source database rows count early and skip further processing of that table. This helps prevent unnecessary connections from accumulating in the pool and prevents wasted effort. For the Nagios plugin, the table is only skipped if *both* databases have empty tables, otherwise it continues to apply validation checks and will fail early as before. refs GH-20
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I recreated the source MySQL database in a test LXD container, but initially forgot to insert the test data. When I ran the prototype
mysql2sqlite
binary I got this output (debug logging):and the application went no further.
The Nagios plugin currently applies a rows count check to both source and mirror databases and exits early if there isn't a match. The main app does check for zero rows, but its later in the code and just before attempting the fetch/insert process.
I'm going to defer fixing this until #19 has been completed so I can track the specific problem instead of continuing to tweak/refine the current code before it ever lands here.
The text was updated successfully, but these errors were encountered: