Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x-pack/filebeat/input/awss3 ; Fix nil hit panic when a getter is invoked on input metric #36101

Merged
merged 13 commits into from
Jul 24, 2023
Merged
2 changes: 1 addition & 1 deletion x-pack/filebeat/docs/inputs/input-aws-s3.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -797,7 +797,7 @@ observe the activity of the input.
| `sqs_messages_inflight_gauge` | Number of SQS messages inflight (gauge).
| `sqs_messages_returned_total` | Number of SQS message returned to queue (happens on errors implicitly after visibility timeout passes).
| `sqs_messages_deleted_total` | Number of SQS messages deleted.
| `sqs_messages_waiting_gauge` | Number of SQS messages waiting in the SQS queue (gauge). The value is refreshed every minute via data from GetQueueAttributes. A value of (0) indicates `no messages waiting` but a value 0f (-1) indicates `metrics not collected/error collecting metrics `
| `sqs_messages_waiting_gauge` | Number of SQS messages waiting in the SQS queue (gauge). The value is refreshed every minute via data from https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQueueAttributes.html<GetQueueAttributes>. A value of `-1` indicates the metric is uninitialized or could not be collected due to an error.
| `sqs_worker_utilization` | Rate of SQS worker utilization over previous 5 seconds. 0 indicates idle, 1 indicates all workers utilized.
| `sqs_message_processing_time` | Histogram of the elapsed SQS processing times in nanoseconds (time of receipt to time of delete/return).
| `sqs_lag_time` | Histogram of the difference between the SQS SentTimestamp attribute and the time when the SQS message was received expressed in nanoseconds.
Expand Down
4 changes: 4 additions & 0 deletions x-pack/filebeat/input/awss3/input.go
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,10 @@ func getProviderFromDomain(endpoint string, ProviderOverride string) string {
func pollSqsWaitingMetric(ctx context.Context, receiver *sqsReader) {
t := time.NewTicker(time.Minute)
defer t.Stop()
// Initialize the metric to 0 at the start of the one minute time interval to avoid
// giving misleading metric value -1 even though SQS messages are processed.
// The value will be updated every minute
receiver.metrics.sqsMessagesWaiting.Set(int64(0))
bhapas marked this conversation as resolved.
Show resolved Hide resolved
for {
select {
case <-ctx.Done():
Expand Down
3 changes: 0 additions & 3 deletions x-pack/filebeat/input/awss3/sqs.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,6 @@ func (r *sqsReader) Receive(ctx context.Context) error {
// Process each SQS message asynchronously with a goroutine.
r.log.Debugf("Received %v SQS messages.", len(msgs))
r.metrics.sqsMessagesReceivedTotal.Add(uint64(len(msgs)))
// Initialize the sqs_message_waiting_guage to 0 to indicate that that SQS messages are received.
// PollSqsWaitingMetric shall reassign the value if there are messages waiting or if there is an error in processing the messages.
r.metrics.sqsMessagesWaiting.Set(int64(0))
workerWg.Add(len(msgs))

for _, msg := range msgs {
Expand Down