Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[epic] flaky tests #37306

Open
thaJeztah opened this issue Jun 18, 2018 · 5 comments
Open

[epic] flaky tests #37306

thaJeztah opened this issue Jun 18, 2018 · 5 comments
Labels
area/testing kind/epic Epics to track work on related tickets status/needs-attention Calls for a collective discussion during a review session

Comments

@thaJeztah
Copy link
Member

thaJeztah commented Jun 18, 2018 •
edited
Loading

This is a list of tests that have shown to be flakey on master. This list is collected from this gist; https://gist.github.com/tonistiigi/9e91cbb968dea83113d09ce5f9dbff79 (generated from logs since January; some of these may already have been resolved)

Status Count Test Tracking-issue
✅ #37315 443 DockerDaemonSuite.TestDaemonNoSpaceLeftOnDeviceError #37277
130 DockerSwarmSuite.TestAPIServiceUpdatePort #36501 / #30427 / #36743
129 DockerSwarmSuite.TestSwarmPublishDuplicatePorts #36501 / #30427 / #36743
✅ #36750 124 TestMount #34321 note: 100% of failures is on rhel-74-selinux-overlay2-stable
119 DockerSuite.TestRunSeccompUnconfinedCloneUserns note: 100% of failures is on centos-7-noselinux-devicemapper
119 DockerSuite.TestRunSeccompAllowPrivCloneUserns note: 100% of failures is on centos-7-noselinux-devicemapper
✅ #35336 119 DockerDaemonSuite.TestDaemonUserNamespaceRootSetting #35336 note: 100% of failures is on centos-7-noselinux-devicemapper
116 DockerDaemonSuite.TestRemoveContainerAfterLiveRestore note: 100% of failures is on rhel72 && overlay
✅ #35998 75 TestCreateServiceMultipleTimes related: #35998
✅ #36502 59 DockerSwarmSuite.TestServiceUpdatePort #36501 / #30427 / #36743 test was removed
55 DockerSwarmSuite.TearDownTest #33041 / #33805
44 TestServicePlugin #37132
41 DockerSwarmSuite.TestAPISwarmLeaderElection #32673 (related: #36788)
39 TestCreateServiceSecretFileMode #37132
35 DockerSwarmSuite.TestSwarmClusterRotateUnlockKey #38885 (related: #33041)
33 TestCreateServiceConfigFileMode #36907
26 DockerSwarmSuite.TestAPISwarmServicesCreate #36903
21 DockerSwarmSuite.TestSwarmLockUnlockCluster #34051 / #35689
20 DockerSwarmSuite.TestAPISwarmLeaderProxy #32562
20 DockerSuite.TestRunOOMExitCode 100% of failures on CentOS and RHEL. Related: #36540
✅ #37438 18 DockerDaemonSuite.TestRunContainerWithBridgeNone #37246
16 DockerSwarmSuite.TestSwarmNetworkPluginV2 -
16 DockerSuite.TestEventsOOMDisableTrue related: #37220
15 DockerSwarmSuite.TestAPISwarmRaftQuorum #34988 / #35039
14 DockerNetworkSuite.TestDockerPluginV2NetworkDriver 100% of failures on RHEL 7.2 + overlay (related: #28305)
9 TestServiceRemoveKeepsIngressNetwork #39426 100% of failures on Ubuntu 16.04 + ZFS
9 DockerSwarmSuite.TestAPISwarmServicesMultipleAgents #23626
✅ #36551 8 TestServiceWithPredefinedNetwork #36547
7 DockerSwarmSuite.TestAPISwarmNodeDrainPause #23516
7 DockerSuite.TestExecInteractiveStdinClose #36877
6 DockerSwarmSuite.TestAPISwarmManagerRestore #34993
6 DockerSuite.TestLogsTail -
6 DockerSuite.TestLogsContainerSmallerThanPage -
5 TestIsCopyFileRangeSyscallAvailable 100% of failures on rhel-74-selinux-overlay2-stable
5 DockerSuite.TestLogsContainerBiggerThanPage -
4 TestUpdateMemory -
4 DockerSwarmSuite.TestSwarmJoinPromoteLocked #35263
✅ 4 DockerSwarmSuite.TestAPISwarmServicesUpdateStartFirst #35775 / #35825 #35825 (comment)
4 DockerSuite.TestRestartContainerwithRestartPolicy #32219 100% on rhel72 && overlay
4 DockerSuite.TestLogsTimestamps -
✅ 4 DockerSuite.TestLogsContainerMuchBiggerThanPage #36500 / #36168
✅ #35982 3 TestDaemonRestartKillContainers #35914
#27466 3 DockerSwarmSuite.TestSwarmContainerAutoStart #26588 #29663
✅ #25077 3 DockerSwarmSuite.TestAPISwarmRestartCluster #24590

Other failures:

Count Failure Notes
25 failed to get layer 100% of failures on Ubuntu 16.04, either with BTRFS or ZFS
10 access denied 100% of failures on Ubuntu 16.04, either with AUFS, BTRFS or ZFS
6 GIT_SHA1: unbound variable 100% of failures on Ubuntu 16.04, either with AUFS, BTRFS or ZFS
@thaJeztah
Copy link
Member Author

/cc @vdemeester @AntaresS @tiborvass

@thaJeztah thaJeztah added status/needs-attention Calls for a collective discussion during a review session area/testing labels Jun 18, 2018
This was referenced Jul 6, 2018
Closed
Merged
@johnstep johnstep mentioned this issue Aug 16, 2018
Merged
@kolyshkin
Copy link
Contributor

kolyshkin commented Aug 28, 2018 •
edited
Loading

TestMount should be fixed by #36750 (and I can't update the description)

@kolyshkin
Copy link
Contributor

DockerSwarmSuite.TearDownTest is not a test case per se, but rather a cleanup after each test, so when showing a failure from it, one needs to see which test case is causing it -- the problem is with that test case, not the TearDownTest

@thaJeztah
Copy link
Member Author

@kolyshkin thanks! updated 👍

@olljanat
Copy link
Contributor

olljanat commented Jan 8, 2019

TestServiceWithDefaultAddressPoolInit looks to be also flaky. Created item #38514

@olljanat olljanat mentioned this issue Jan 14, 2019
Merged
@LinuxMercedes LinuxMercedes mentioned this issue Jan 15, 2019
Merged
@olljanat olljanat mentioned this issue Jan 19, 2019
Closed
@thaJeztah thaJeztah mentioned this issue Mar 16, 2019
Closed
@LinuxMercedes LinuxMercedes mentioned this issue Apr 23, 2019
Merged
@djsweet djsweet mentioned this issue Apr 30, 2019
Closed
@thaJeztah thaJeztah mentioned this issue Jun 21, 2019
Merged
@thaJeztah thaJeztah mentioned this issue Jul 2, 2019
Open
@dperny dperny mentioned this issue Jul 26, 2019
Merged
This was referenced Sep 1, 2019
Merged
Merged
@thaJeztah thaJeztah changed the title EPIC: flaky tests [epic] flaky tests Mar 7, 2023
@thaJeztah thaJeztah added the kind/epic Epics to track work on related tickets label Mar 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/testing kind/epic Epics to track work on related tickets status/needs-attention Calls for a collective discussion during a review session
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@thaJeztah @kolyshkin @olljanat