Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DATA-fix] Attempt to reduce false negatives in sync/collector tests. #1727

Merged
merged 5 commits into from
Jan 3, 2023
Merged

[DATA-fix] Attempt to reduce false negatives in sync/collector tests. #1727

merged 5 commits into from
Jan 3, 2023

Conversation

AaronCasas
Copy link
Contributor

Saw flakiness in these tests in these runs. Was not able to reproduce locally:

builtin$ go test -race --count=200 -run TestDataCaptureUpload/If_tabular_uploads_fail_transiently,_they_should_be_retried_until_they_succeed
PASS
ok      go.viam.com/rdk/services/datamanager/builtin    115.741s

data$ go test -race --count=2000 -run TestClose
PASS
ok      go.viam.com/rdk/data    112.579s

This makes 3 changes:

  • Reduces test collector interval from 15ms -> 5ms. Since the test is essentially checking "after + , did we capture stuff?", decreasing the expected interval from 15->5ms and increasing the leeway from 10->20ms should reduce the chances of this happening.
  • In the failing TestDataCaptureUpload test we were reassigning the syncTime from 150ms -> 100ms. We don't need to do that. In the failures, sync requests were being cancelled before they were finished. Increasing the time they have to run should reduce the chance of that.
  • Also removes a big comment that I forgot to remove in a previous PR.

@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Jan 3, 2023
agreenb
agreenb approved these changes Jan 3, 2023
View reviewed changes
Copy link
Contributor

@agreenb agreenb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 3, 2023
@agreenb
Copy link
Contributor

agreenb commented Jan 3, 2023

Thanks again for the super quick fixes and cleanup! 🚀

Looks like still running into a flaky collector test, so request to update the PR description accordingly once that's bumped.

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 3, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 3, 2023
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 3, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jan 3, 2023

Code Coverage

Note: merge base coverage results not available, comparing against closest c140995~1=(6a3508a) instead

Package Line Rate Delta Health
go.viam.com/rdk/components/arm 56% 0.00% ➖
go.viam.com/rdk/components/arm/universalrobots 41% 0.00% ❌
go.viam.com/rdk/components/arm/xarm 2% 0.00% ❌
go.viam.com/rdk/components/arm/yahboom 7% 0.00% ❌
go.viam.com/rdk/components/audioinput 55% 0.00% ➖
go.viam.com/rdk/components/base 63% 0.00% ➖
go.viam.com/rdk/components/base/agilex 62% 0.00% ➖
go.viam.com/rdk/components/base/boat 41% 0.00% ❌
go.viam.com/rdk/components/base/wheeled 76% 0.00% ✅
go.viam.com/rdk/components/board 68% 0.00% ➖
go.viam.com/rdk/components/board/arduino 10% 0.00% ❌
go.viam.com/rdk/components/board/commonsysfs 47% 0.00% ❌
go.viam.com/rdk/components/board/fake 38% 0.00% ❌
go.viam.com/rdk/components/board/numato 19% 0.00% ❌
go.viam.com/rdk/components/board/pi 50% 0.00% ➖
go.viam.com/rdk/components/camera 66% +0.26% ➖
go.viam.com/rdk/components/camera/align 64% +0.48% ➖
go.viam.com/rdk/components/camera/fake 67% 0.00% ➖
go.viam.com/rdk/components/camera/ffmpeg 77% 0.00% ✅
go.viam.com/rdk/components/camera/transformpipeline 77% -1.00% ✅
go.viam.com/rdk/components/camera/videosource 49% -0.63% ❌
go.viam.com/rdk/components/encoder/fake 77% 0.00% ✅
go.viam.com/rdk/components/gantry 60% 0.00% ➖
go.viam.com/rdk/components/gantry/multiaxis 84% 0.00% ✅
go.viam.com/rdk/components/gantry/oneaxis 86% 0.00% ✅
go.viam.com/rdk/components/generic 83% 0.00% ✅
go.viam.com/rdk/components/gripper 68% 0.00% ➖
go.viam.com/rdk/components/input 87% 0.00% ✅
go.viam.com/rdk/components/input/gpio 84% 0.00% ✅
go.viam.com/rdk/components/motor 77% 0.00% ✅
go.viam.com/rdk/components/motor/dimensionengineering 63% 0.00% ➖
go.viam.com/rdk/components/motor/dmc4000 69% 0.00% ➖
go.viam.com/rdk/components/motor/fake 57% 0.00% ➖
go.viam.com/rdk/components/motor/gpio 64% 0.00% ➖
go.viam.com/rdk/components/motor/gpiostepper 57% 0.00% ➖
go.viam.com/rdk/components/motor/tmcstepper 62% 0.00% ➖
go.viam.com/rdk/components/movementsensor 75% 0.00% ✅
go.viam.com/rdk/components/movementsensor/cameramono 40% 0.00% ❌
go.viam.com/rdk/components/movementsensor/gpsnmea 37% 0.00% ❌
go.viam.com/rdk/components/movementsensor/gpsrtk 28% 0.00% ❌
go.viam.com/rdk/components/posetracker 86% 0.00% ✅
go.viam.com/rdk/components/sensor 86% 0.00% ✅
go.viam.com/rdk/components/sensor/ultrasonic 34% 0.00% ❌
go.viam.com/rdk/components/servo 68% 0.00% ➖
go.viam.com/rdk/components/servo/gpio 71% 0.00% ✅
go.viam.com/rdk/config 76% 0.00% ✅
go.viam.com/rdk/control 57% 0.00% ➖
go.viam.com/rdk/data 79% 0.00% ✅
go.viam.com/rdk/examples/customresources/demos/complexmodule 0% 0.00% ❌
go.viam.com/rdk/examples/customresources/demos/remoteserver 0% 0.00% ❌
go.viam.com/rdk/grpc 25% 0.00% ❌
go.viam.com/rdk/ml 67% 0.00% ➖
go.viam.com/rdk/ml/inference 70% 0.00% ✅
go.viam.com/rdk/module 64% 0.00% ➖
go.viam.com/rdk/module/modmanager 79% 0.00% ✅
go.viam.com/rdk/motionplan 71% +11.40% ✅
go.viam.com/rdk/octree 98% 0.00% ✅
go.viam.com/rdk/operation 82% 0.00% ✅
go.viam.com/rdk/pointcloud 72% +0.52% ✅
go.viam.com/rdk/protoutils 59% 0.00% ➖
go.viam.com/rdk/referenceframe 70% 0.00% ✅
go.viam.com/rdk/registry 89% 0.00% ✅
go.viam.com/rdk/resource 84% 0.00% ✅
go.viam.com/rdk/rimage 78% 0.00% ✅
go.viam.com/rdk/rimage/depthadapter 94% 0.00% ✅
go.viam.com/rdk/rimage/transform 73% 0.00% ✅
go.viam.com/rdk/rimage/transform/cmd/extrinsic_calibration 67% 0.00% ➖
go.viam.com/rdk/robot 85% 0.00% ✅
go.viam.com/rdk/robot/client 82% 0.00% ✅
go.viam.com/rdk/robot/framesystem 68% 0.00% ➖
go.viam.com/rdk/robot/impl 81% 0.00% ✅
go.viam.com/rdk/robot/server 59% 0.00% ➖
go.viam.com/rdk/robot/web 62% 0.00% ➖
go.viam.com/rdk/robot/web/stream 87% 0.00% ✅
go.viam.com/rdk/services/armremotecontrol 71% 0.00% ✅
go.viam.com/rdk/services/armremotecontrol/builtin 52% 0.00% ➖
go.viam.com/rdk/services/baseremotecontrol 71% 0.00% ✅
go.viam.com/rdk/services/baseremotecontrol/builtin 79% 0.00% ✅
go.viam.com/rdk/services/datamanager 79% 0.00% ✅
go.viam.com/rdk/services/datamanager/builtin 78% 0.00% ✅
go.viam.com/rdk/services/datamanager/datacapture 70% 0.00% ✅
go.viam.com/rdk/services/datamanager/datasync 0% 0.00% ❌
go.viam.com/rdk/services/motion 63% 0.00% ➖
go.viam.com/rdk/services/motion/builtin 88% 0.00% ✅
go.viam.com/rdk/services/navigation 53% 0.00% ➖
go.viam.com/rdk/services/sensors 77% 0.00% ✅
go.viam.com/rdk/services/sensors/builtin 97% 0.00% ✅
go.viam.com/rdk/services/shell 14% 0.00% ❌
go.viam.com/rdk/services/slam 84% 0.00% ✅
go.viam.com/rdk/services/slam/builtin 73% 0.00% ✅
go.viam.com/rdk/services/vision 80% 0.00% ✅
go.viam.com/rdk/services/vision/builtin 74% 0.00% ✅
go.viam.com/rdk/session 97% 0.00% ✅
go.viam.com/rdk/spatialmath 84% 0.00% ✅
go.viam.com/rdk/subtype 92% 0.00% ✅
go.viam.com/rdk/utils 72% 0.00% ✅
go.viam.com/rdk/vision 26% 0.00% ❌
go.viam.com/rdk/vision/chess 80% 0.00% ✅
go.viam.com/rdk/vision/delaunay 87% 0.00% ✅
go.viam.com/rdk/vision/keypoints 92% 0.00% ✅
go.viam.com/rdk/vision/objectdetection 82% 0.00% ✅
go.viam.com/rdk/vision/odometry 60% 0.00% ➖
go.viam.com/rdk/vision/odometry/cmd 0% 0.00% ❌
go.viam.com/rdk/vision/segmentation 49% 0.00% ❌
go.viam.com/rdk/web/server 25% 0.00% ❌
Summary 66% (21020 / 31689) +0.72% ➖

@AaronCasas AaronCasas merged commit 9f6800b into viamrobotics:main Jan 3, 2023
@AaronCasas AaronCasas deleted the data-fix-flaky-test branch January 3, 2023 20:17
@bazile-clyde bazile-clyde mentioned this pull request Feb 7, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@agreenb agreenb agreenb approved these changes

Assignees
No one assigned
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AaronCasas @agreenb @viambot