Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DATA-566] Fix Flaky Test in DetectionSource #1512

Merged
merged 4 commits into from
Oct 21, 2022
Merged

[DATA-566] Fix Flaky Test in DetectionSource #1512

merged 4 commits into from
Oct 21, 2022

Conversation

kharijarrett
Copy link
Member

No description provided.

kharijarrett
kharijarrett commented Oct 21, 2022
View reviewed changes
Copy link
Member Author

@kharijarrett kharijarrett left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi okay so the fix here was to add another case to NextResult that times out after 2 seconds and makes the return nil. I changed the test to include a timer that checks to make sure that behavior is what's expected (time out after 2 seconds).
I ran this 5000 times in a canon-shell-amd64 and it passed (and before the change it broke after ~1000 attempts)

@kharijarrett kharijarrett marked this pull request as ready for review October 21, 2022 14:43
bhaney
bhaney reviewed Oct 21, 2022
View reviewed changes
Copy link
Member

@bhaney bhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one question for the added for-loop, otherwise looking good

vision/objectdetection/detection_source.go Outdated
Comment on lines 108 to 115

for res == nil {
res, err = s.NextResult(ctx)
if err != nil {
return nil, nil, err
}
}

Copy link
Member

@bhaney bhaney Oct 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] Do you still need this for loop? Is NextResult ever going to produce a nil result now (and not an error)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naw I guess we don't... removing now. Gonna run the test a few thousand times again but you're right.

@github-actions
Copy link
Contributor

Code Coverage

Package Line Rate Health
go.viam.com/rdk/components/arm 59% ➖
go.viam.com/rdk/components/arm/universalrobots 12% ❌
go.viam.com/rdk/components/arm/xarm 2% ❌
go.viam.com/rdk/components/arm/yahboom 7% ❌
go.viam.com/rdk/components/audioinput 55% ➖
go.viam.com/rdk/components/base 68% ➖
go.viam.com/rdk/components/base/agilex 62% ➖
go.viam.com/rdk/components/base/boat 41% ❌
go.viam.com/rdk/components/base/wheeled 76% ✅
go.viam.com/rdk/components/board 69% ➖
go.viam.com/rdk/components/board/arduino 10% ❌
go.viam.com/rdk/components/board/commonsysfs 47% ❌
go.viam.com/rdk/components/board/fake 39% ❌
go.viam.com/rdk/components/board/numato 19% ❌
go.viam.com/rdk/components/board/pi 50% ➖
go.viam.com/rdk/components/camera 66% ➖
go.viam.com/rdk/components/camera/fake 67% ➖
go.viam.com/rdk/components/camera/ffmpeg 72% ✅
go.viam.com/rdk/components/camera/transformpipeline 81% ✅
go.viam.com/rdk/components/camera/videosource 56% ➖
go.viam.com/rdk/components/encoder/fake 77% ✅
go.viam.com/rdk/components/gantry 68% ➖
go.viam.com/rdk/components/gantry/multiaxis 84% ✅
go.viam.com/rdk/components/gantry/oneaxis 86% ✅
go.viam.com/rdk/components/generic 85% ✅
go.viam.com/rdk/components/gripper 82% ✅
go.viam.com/rdk/components/input 86% ✅
go.viam.com/rdk/components/input/gpio 87% ✅
go.viam.com/rdk/components/motor 82% ✅
go.viam.com/rdk/components/motor/dmc4000 69% ➖
go.viam.com/rdk/components/motor/fake 60% ➖
go.viam.com/rdk/components/motor/gpio 65% ➖
go.viam.com/rdk/components/motor/gpiostepper 59% ➖
go.viam.com/rdk/components/motor/tmcstepper 66% ➖
go.viam.com/rdk/components/movementsensor 67% ➖
go.viam.com/rdk/components/movementsensor/cameramono 39% ❌
go.viam.com/rdk/components/movementsensor/gpsnmea 37% ❌
go.viam.com/rdk/components/movementsensor/gpsrtk 28% ❌
go.viam.com/rdk/components/posetracker 88% ✅
go.viam.com/rdk/components/sensor 88% ✅
go.viam.com/rdk/components/sensor/ultrasonic 31% ❌
go.viam.com/rdk/components/servo 77% ✅
go.viam.com/rdk/config 77% ✅
go.viam.com/rdk/control 57% ➖
go.viam.com/rdk/data 78% ✅
go.viam.com/rdk/grpc 25% ❌
go.viam.com/rdk/ml 67% ➖
go.viam.com/rdk/ml/inference 70% ✅
go.viam.com/rdk/motionplan 69% ➖
go.viam.com/rdk/operation 84% ✅
go.viam.com/rdk/pointcloud 71% ✅
go.viam.com/rdk/protoutils 62% ➖
go.viam.com/rdk/referenceframe 78% ✅
go.viam.com/rdk/registry 88% ✅
go.viam.com/rdk/resource 85% ✅
go.viam.com/rdk/rimage 78% ✅
go.viam.com/rdk/rimage/depthadapter 94% ✅
go.viam.com/rdk/rimage/transform 73% ✅
go.viam.com/rdk/rimage/transform/cmd/extrinsic_calibration 67% ➖
go.viam.com/rdk/robot 93% ✅
go.viam.com/rdk/robot/client 79% ✅
go.viam.com/rdk/robot/framesystem 68% ➖
go.viam.com/rdk/robot/impl 79% ✅
go.viam.com/rdk/robot/server 58% ➖
go.viam.com/rdk/robot/web 60% ➖
go.viam.com/rdk/robot/web/stream 87% ✅
go.viam.com/rdk/services/armremotecontrol 75% ✅
go.viam.com/rdk/services/armremotecontrol/builtin 25% ❌
go.viam.com/rdk/services/baseremotecontrol 75% ✅
go.viam.com/rdk/services/baseremotecontrol/builtin 71% ✅
go.viam.com/rdk/services/datamanager 62% ➖
go.viam.com/rdk/services/datamanager/builtin 78% ✅
go.viam.com/rdk/services/datamanager/datacapture 34% ❌
go.viam.com/rdk/services/datamanager/datasync 70% ➖
go.viam.com/rdk/services/motion 68% ➖
go.viam.com/rdk/services/motion/builtin 89% ✅
go.viam.com/rdk/services/navigation 54% ➖
go.viam.com/rdk/services/sensors 78% ✅
go.viam.com/rdk/services/sensors/builtin 97% ✅
go.viam.com/rdk/services/shell 15% ❌
go.viam.com/rdk/services/slam 86% ✅
go.viam.com/rdk/services/slam/builtin 73% ✅
go.viam.com/rdk/services/vision 82% ✅
go.viam.com/rdk/services/vision/builtin 74% ✅
go.viam.com/rdk/spatialmath 85% ✅
go.viam.com/rdk/subtype 96% ✅
go.viam.com/rdk/utils 71% ✅
go.viam.com/rdk/vision 26% ❌
go.viam.com/rdk/vision/chess 80% ✅
go.viam.com/rdk/vision/delaunay 87% ✅
go.viam.com/rdk/vision/keypoints 92% ✅
go.viam.com/rdk/vision/objectdetection 82% ✅
go.viam.com/rdk/vision/odometry 60% ➖
go.viam.com/rdk/vision/odometry/cmd 0% ❌
go.viam.com/rdk/vision/segmentation 49% ❌
go.viam.com/rdk/web/server 26% ❌
Summary 66% (19030 / 28764) ➖

bhaney
bhaney approved these changes Oct 21, 2022
View reviewed changes
Copy link
Member

@bhaney bhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the flakiness doesn't come back after a few thousand test runs, then this LGTM

@kharijarrett kharijarrett merged commit f1810e6 into viamrobotics:main Oct 21, 2022
@kharijarrett kharijarrett deleted the DetSrcFlakyTest branch November 14, 2022 18:44
@bazile-clyde bazile-clyde mentioned this pull request Feb 7, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bhaney bhaney bhaney approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kharijarrett @bhaney