Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve apply_dml_events_validation batch size handling #10

Conversation

jjh-kim
Copy link
Contributor

@jjh-kim jjh-kim commented Jul 25, 2024

Fixed two major issues of apply_dml_events_validation

  • When a larger number (higher than apply_dml_events_validation_batch_size) of DML events occur within a given time range, the validator attempts to cut the batch size by reducing the range size. However, when the time range is already 1 second, it can’t reduce the time range further because the binlog event timestamp has a 1-second resolution, which causes it to fall into an infinite loop.
  • When creating timestamp ranges (line 282) the while loop condition was batch_start_timestamp < end_timestamp. This had possibility of skipping the end_timestamp when the batch_duration is evaluated as 0

Additionally added feature to not run full_dml_event_validation if full_dml_event_validation is set to 0

@jjh-kim jjh-kim merged commit 58c8fa4 into develop Jul 25, 2024
2 checks passed
@jjh-kim jjh-kim added the bug Something isn't working label Jul 25, 2024
@jjh-kim jjh-kim deleted the bugfix/improve-apply-dml-events-validation-batch-size-handling branch July 25, 2024 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant