Add API to return automatic compaction config history #13699
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
suneet-s
commented
Jan 20, 2023
server/src/main/java/org/apache/druid/server/coordinator/DataSourceCompactionConfigHistory.java
Show resolved
Hide resolved
kfaraz
approved these changes
Jan 21, 2023
Comment on lines
74
to
88
| if (newEntry != null || (current != null && !hasDataSourceCompactionConfig)) { | ||
| if (newEntry == null) { | ||
| newEntry = new DatasourceCompactionConfigAuditEntry( | ||
| new DatasourceCompactionConfigAuditEntry.GlobalCompactionConfig( | ||
| coordinatorCompactionConfig.getCompactionTaskSlotRatio(), | ||
| coordinatorCompactionConfig.getMaxCompactionTaskSlots(), | ||
| coordinatorCompactionConfig.isUseAutoScaleSlots() | ||
| ), | ||
| null, | ||
| auditInfo, | ||
| auditTime | ||
| ); | ||
| } | ||
| auditEntries.push(newEntry); | ||
| } |
There was a problem hiding this comment.
Nit: maybe simplify this for readability:
if (newEntry != null) {
auditEntries.push(newEntry);
} else if (current != null && !hasDataSourceCompactionConfig) {
newEntry = ...
auditEntries.push(newEntry);
}| /** | ||
| * A DTO containing audit information for compaction config for a datasource. | ||
| */ | ||
| public class DatasourceCompactionConfigAuditEntry |
There was a problem hiding this comment.
Nit: Rename to DataSourceCompactionConfigAuditEntry with a capitalized S in DataSource, similar to rest of the classes here.
maytasm
reviewed
Jan 23, 2023
| count | ||
| ); | ||
| } else { | ||
| auditEntries = auditManager.fetchAuditHistory( |
There was a problem hiding this comment.
Can a user pass in both interval and count?
There was a problem hiding this comment.
I copied the pattern in CoordinatorDynamicConfigsResource#getDatasourceRuleHistory where it looks like we ignore the count silently if the interval is specified.
maytasm
approved these changes
Jan 24, 2023
abhagraw
pushed a commit
to abhagraw/druid
that referenced
this pull request
Feb 8, 2023
Add a new API to return the history of changes to automatic compaction config history to make it easy for users to see what changes have been made to their auto-compaction config. The API is scoped per dataSource to allow users to triage issues with an individual dataSource. The API responds with a list of configs when there is a change to either the settings that impact all auto-compaction configs on a cluster or the dataSource in question.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add a new API to return the history of changes to automatic compaction config history to make it easy for users to see what changes have been made to their auto-compaction config.
The API is scoped per dataSource to allow users to triage issues with an individual dataSource. The API responds with a list of configs when there is a change to either the settings that impact all auto-compaction configs on a cluster or the dataSource in question.
I had considered adding another API to expose the entire compaction config history as it was stored in the audit table, however I chose against doing this, as I didn't think users would want the ability to see this global view.
Release note
New: You can now get the history of changes to automatic compaction config for a dataSource via an API.
Key changed/added classes in this PR
DataSourceCompactionConfigHistory#add- Ensures that only changes that affect a particular dataSource's auto-compaction config are added to the historyCoordinatorCompactionConfigsResource#getCompactionConfigHistory- the API endpointThis PR has: