Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REPLAY-1902 Make SR uploader even more eager #1377

Merged
merged 1 commit into from
Jul 19, 2023

Conversation

ncreated
Copy link
Member

What and why?

⏱️ Following on #1363, we want to make SR uploader even more eager.

The rationale is to leverage -beta and to promote data consistency over uploads performance slightly more. After getting numbers with next observability tasks, we will lower the preset to more optimised one.

How?

We're leveraging the performance preset override API added earlier. I selected new values by testing in example project - playing smooth scenario and adjusting numbers to achieve relatively short batch buffer.

Review checklist

  • Feature or bugfix MUST have appropriate tests (unit, integration)
  • Make sure each commit and the PR mention the Issue number or JIRA reference
  • Add CHANGELOG entry for user facing changes

Custom CI job configuration (optional)

  • Run unit tests
  • Run integration tests
  • Run smoke tests

by overwriting performance presset more aggressively.
@ncreated ncreated self-assigned this Jul 18, 2023
Comment on lines -70 to +75
meanFileAge: 5, // equivalent of `batchSize: .small` - see `DatadogCore.PerformancePreset`
minUploadDelay: 1 // equivalent of `uploadFrequency: .frequent` - see `DatadogCore.PerformancePreset`
meanFileAge: 2, // vs 5s with `batchSize: .small` - see `DatadogCore.PerformancePreset`
uploadDelay: (
initial: 2, // vs 5s with `uploadFrequency: .frequent`
range: 0.6..<6, // vs 1s ..< 10s with `uploadFrequency: .frequent`
changeRate: 0.75 // vs 0.1 with `uploadFrequency: .frequent`
)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 For insights how these numbers translate into actual timings, see:

internal struct DataUploadDelay: Delay {
private let minDelay: TimeInterval
private let maxDelay: TimeInterval
private let changeRate: Double
private var delay: TimeInterval
init(performance: UploadPerformancePreset) {
self.minDelay = performance.minUploadDelay
self.maxDelay = performance.maxUploadDelay
self.changeRate = performance.uploadDelayChangeRate
self.delay = performance.initialUploadDelay
}
var current: TimeInterval { delay }
mutating func decrease() {
delay = max(minDelay, delay * (1.0 - changeRate))
}
mutating func increase() {
delay = min(delay * (1.0 + changeRate), maxDelay)
}

@ncreated ncreated marked this pull request as ready for review July 18, 2023 16:42
@ncreated ncreated requested a review from a team as a code owner July 18, 2023 16:42
@ncreated ncreated merged commit 77a18da into develop Jul 19, 2023
@ncreated ncreated deleted the ncreated/REPLAY-1902/sr-uploader-even-more-eager branch July 19, 2023 10:18
@ncreated ncreated mentioned this pull request Jul 25, 2023
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants