Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: corrupted rsources stats captured by processor for dropped jobs #3999

Merged
merged 1 commit into from
Oct 19, 2023

Conversation

atzoum
Copy link
Contributor

@atzoum atzoum commented Oct 19, 2023

Description

Using a unique job id instead of a zero (0) value for every dropped job so that stats are captured correctly.

Problem

By using the same (0) job id, in_count was being incremented for all different stat key combinations (job_run_id, task_run_id, source_id & destination_id), but failed_count was being erroneously incremented always for the first stat key, due to a corrupted jobIdsToStatKeyIndex.

Linear Ticket

Resolves PIPE-433

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@codecov
Copy link

codecov bot commented Oct 19, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (2d43a0c) 72.38% compared to head (49b376f) 70.39%.
Report is 1 commits behind head on release/1.15.x.

Additional details and impacted files
@@                Coverage Diff                 @@
##           release/1.15.x    #3999      +/-   ##
==================================================
- Coverage           72.38%   70.39%   -2.00%     
==================================================
  Files                 368      357      -11     
  Lines               55595    53686    -1909     
==================================================
- Hits                40244    37792    -2452     
- Misses              13068    13624     +556     
+ Partials             2283     2270      -13     
Files Coverage Δ
processor/processor.go 88.31% <100.00%> (-4.14%) ⬇️

... and 47 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@atzoum atzoum merged commit e7b829d into release/1.15.x Oct 19, 2023
43 of 44 checks passed
@atzoum atzoum deleted the fix.rsourcesStats branch October 19, 2023 18:08
koladilip added a commit that referenced this pull request Oct 25, 2023
* chore: advertise gzip support to transformer through X-Feature-Gzip-Support header (#3990)

* chore: release 1.15.2 (#3992)

* fix: update error parsing of eloqua (#3996)

* fix(eloqua): change null values to empty strings inspite of "null"

* address comments

* add logic to consider some warning statusCode as success

* fix lint issues

* fix: corrupted rsources stats captured by processor for dropped jobs (#3999)

* chore: release 1.15.3 (#3998)

* fix: invalid memory address or nil pointer dereference in googlecloudfunction

* fix: merge error blocks in gcf

* chore: add test case for body close

* chore: add test case for body close

---------

Co-authored-by: Gauravudia <60897972+Gauravudia@users.noreply.github.com>
Co-authored-by: devops-github-rudderstack <88187154+devops-github-rudderstack@users.noreply.github.com>
Co-authored-by: Sudip Paul <67197965+ItsSudip@users.noreply.github.com>
Co-authored-by: Aris Tzoumas <atzoumas@rudderstack.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants