-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[crmsh-4.6] Fix: report.utils: Fix the performance issue (bsc#1232821) #1605
[crmsh-4.6] Fix: report.utils: Fix the performance issue (bsc#1232821) #1605
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
368d969
to
13b4f9e
Compare
13b4f9e
to
58ae942
Compare
19947b6
to
11795f2
Compare
crmsh/report/constants.py
Outdated
@@ -3,7 +3,7 @@ | |||
BIN_CRM = "/usr/sbin/crm" | |||
BIN_COLLECTOR = f"{BIN_CRM} report __collector" | |||
COMPRESS_DATA_FLAG = "COMPRESS CRM_REPORT DATA:::" | |||
LOG_PATTERNS = "CRIT: ERROR: error: warning: crit:" | |||
LOG_PATTERNS = "CRIT: ERROR: WARNING: crit: error: warning:" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of defining these patterns as space-separated string, we can define it directly as a list of strings.
The function `extract_critical_log` can be very slow when processing large log files. The main issue is the inefficiency of the regular expression, which combines multiple wildcards (.*) with alternation (|), leading to excessive backtracking. Additionally, the function reads the entire file into memory, which is not optimal for large files. To improve performance, it is better to use `grep` with the -F option to search for fixed strings.
…dify time No need to check the rest of the files if the from time is greater than the modify time of the file.
11795f2
to
5ee01e1
Compare
The function
extract_critical_log
can be very slow when processing large log files. The main issue is the inefficiency of the regular expression, which combines multiple wildcards (.*) with alternation (|), leading to excessive backtracking. Additionally, the function reads the entire file into memory, which is not optimal for large files.To improve performance, it is better to use
grep
with the -F option to search for fixed strings.And for a sequence of archived log files, check the modify time. No need to check the rest of the files if the from time is greater than the modify time of the file.