Skip to content

Adjust concurrency logic and metrics in the concurrency package, and … #176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 23, 2024
Merged

Adjust concurrency logic and metrics in the concurrency package, and … #176

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions concurrency/metrics.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ func (ch *ConcurrencyHandler) EvaluateAndAdjustConcurrency(resp *http.Response,
// Calculate the cumulative score.
cumulativeScore := weightedRateLimitScore + weightedResponseCodeScore + weightedResponseTimeScore

// Log the feedback from each monitoring function for debugging.
// Detailed debugging output
ch.logger.Debug("Evaluate and Adjust Concurrency",
zap.String("event", "EvaluateConcurrency"),
zap.Float64("weightedRateLimitScore", weightedRateLimitScore),
Expand All @@ -71,6 +71,14 @@ func (ch *ConcurrencyHandler) EvaluateAndAdjustConcurrency(resp *http.Response,
zap.Duration("responseTime", responseTime),
)

// Check for successful response and log appropriately
if responseCodeFeedback == 1 { // Assuming 1 indicates success
ch.logger.Info("Successful response noted, checking for scaling necessity.",
zap.String("API Response", "Success"),
zap.Int("StatusCode", resp.StatusCode),
)
}

// Check critical thresholds
if rateLimitFeedback <= RateLimitCriticalThreshold || weightedResponseCodeScore >= ErrorResponseThreshold {
ch.logger.Warn("Scaling down due to critical threshold breach",
Expand All @@ -82,7 +90,7 @@ func (ch *ConcurrencyHandler) EvaluateAndAdjustConcurrency(resp *http.Response,
return
}

// Evaluate cumulative impact and make a scaling decision.
// Evaluate cumulative impact and make a scaling decision based on the cumulative score and other metrics.
if cumulativeScore < 0 {
utilizedBefore := len(ch.sem) // Tokens in use before scaling down.
ch.ScaleDown()
Expand Down Expand Up @@ -170,9 +178,10 @@ func (ch *ConcurrencyHandler) MonitorServerResponseCodes(resp *http.Response) in
zap.Float64("ErrorRate", errorRate),
)

// Only suggest a scale-down if the error rate exceeds the threshold
if errorRate > ErrorRateThreshold {
return -1
} else if errorRate <= ErrorRateThreshold && len(ch.sem) < MaxConcurrency {
} else if len(ch.sem) < MaxConcurrency {
return 1
}
return 0
Expand Down