Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move Windows CI to Github Actions #2334

Closed
derekbruening opened this issue Dec 7, 2020 · 2 comments · Fixed by #2346
Closed

Move Windows CI to Github Actions #2334

derekbruening opened this issue Dec 7, 2020 · 2 comments · Fixed by #2346
Labels
Component-Tests OpSys-Windows

Comments

@derekbruening
Copy link
Contributor

Xref similar issue for DR: DynamoRIO/dynamorio#4131

With the loss of Travis we have moved our unix CI to Github Actions: #2323.
This covers also moving our Windows CI from Appveyor, to gain parallel builds and auto-prior-build cancellation (DynamoRIO/dynamorio#4585).

My first attempts are in PR #2326. The GA image we're targeting is Server 2016, the same as our attempt to update our Appveyor image in #2250 and PR #2265. Just like there, we hit major failures for 64-bit, due to #2328, which is now fixed; and major 32-bit failures: #2329. Both were DR problems and both are now fixed.

Now we're doing to a slew of test failures:

32-bit:
https://pipelines.actions.githubusercontent.com/tMvJIh3ZIUXIfOP7dfgG7UBvQKftGfP2Bp1gAjRieRAnDKBVqy/_apis/pipelines/1/runs/102/signedlogcontent/4?urlExpires=2020-12-06T21%3A21%3A31.3518373Z&urlSigningMethod=HMACV1&urlSignature=UM25pzPPpL5cFdRg9YFc7W%2B%2B4HA%2BYGaWei0XZiag1Ds%3D

2020-12-06T16:51:28.5125095Z drmemory-dbg-32: 112 tests passed, **** 33 tests failed, but ignoring 4 for i1938: ****
2020-12-06T16:51:28.5125807Z 	procterm 
2020-12-06T16:51:28.5126266Z 	operators 
2020-12-06T16:51:28.5126722Z 	cs2bug 
2020-12-06T16:51:28.5127129Z 	state 
2020-12-06T16:51:28.5127781Z 	procterm.nativeparent 
2020-12-06T16:51:28.5128548Z 	malloc_callstacks 
2020-12-06T16:51:28.5129032Z 	reachable 
2020-12-06T16:51:28.5129567Z 	wrap_malloc 
2020-12-06T16:51:28.5130060Z 	wrap_operators 
2020-12-06T16:51:28.5130598Z 	(ignore: i#1938) 	wrap_cs2bug 
2020-12-06T16:51:28.5131082Z 	wrap_wincrt 
2020-12-06T16:51:28.5131529Z 	wincrt 
2020-12-06T16:51:28.5131977Z 	wrap_wincrtdbg 
2020-12-06T16:51:28.5132462Z 	mallocMD 
2020-12-06T16:51:28.5132922Z 	cs2bugMTdZI 
2020-12-06T16:51:28.5133436Z 	cs2bugMTd 
2020-12-06T16:51:28.5133980Z 	cs2bugMD 
2020-12-06T16:51:28.5134456Z 	wrap_cs2bugMTd 
2020-12-06T16:51:28.5135012Z 	cs2bugMDd 
2020-12-06T16:51:28.5135571Z 	wrap_operatorsMDd 
2020-12-06T16:51:28.5136065Z 	gdi 
2020-12-06T16:51:28.5136520Z 	handle 
2020-12-06T16:51:28.5136996Z 	handle_only 
2020-12-06T16:51:28.5137498Z 	blacklist 
2020-12-06T16:51:28.5138114Z 	(ignore: i#1938) 	pcache-use 
2020-12-06T16:51:28.5138658Z 	nudge 
2020-12-06T16:51:28.5139274Z 	leak_string 
2020-12-06T16:51:28.5140052Z 	(ignore: i#1938) 	app_suite.pattern 
2020-12-06T16:51:28.5140676Z 	(ignore: i#1938) 	app_suite 
2020-12-06T16:51:28.5141631Z 	drsyscall_test => ASSERT FAILURE: ..\tests\framework\drsyscall_client.c:182: false (failed to get syscall return type)
2020-12-06T16:51:28.5144127Z 	strace_test => ASSERT FAILURE: ..\tests\framework\strace_client.c:172: false (failed to get syscall return type)
2020-12-06T16:51:28.5146076Z 	drstrace_unit_tests 
2020-12-06T16:51:28.5146727Z 	umbra_client_faulty_redzone

64-bit:
https://pipelines.actions.githubusercontent.com/tMvJIh3ZIUXIfOP7dfgG7UBvQKftGfP2Bp1gAjRieRAnDKBVqy/_apis/pipelines/1/runs/102/signedlogcontent/5?urlExpires=2020-12-06T21%3A21%3A38.3981463Z&urlSigningMethod=HMACV1&urlSignature=DbjyahRfLoFQaxgIcCUoAJY4UAckE0LAGiBpBv7pclY%3D

2020-12-06T16:35:46.8873843Z drmemory-dbg-64: 78 tests passed, **** 39 tests failed, but ignoring 1 for i1938: ****
2020-12-06T16:35:46.8874401Z 	procterm 
2020-12-06T16:35:46.8874743Z 	malloc 
2020-12-06T16:35:46.8875048Z 	badjmp 
2020-12-06T16:35:46.8875401Z 	multierror 
2020-12-06T16:35:46.8875895Z 	operators 
2020-12-06T16:35:46.8876221Z 	cs2bug 
2020-12-06T16:35:46.8876539Z 	state 
2020-12-06T16:35:46.8876868Z 	winthreads 
2020-12-06T16:35:46.8877344Z 	procterm.nativeparent 
2020-12-06T16:35:46.8877820Z 	malloc_callstacks 
2020-12-06T16:35:46.8878190Z 	reachable 
2020-12-06T16:35:46.8878585Z 	noreplace_realloc 
2020-12-06T16:35:46.8878959Z 	redzone16 
2020-12-06T16:35:46.8879326Z 	redzone1024 
2020-12-06T16:35:46.8879672Z 	suppress 
2020-12-06T16:35:46.8880087Z 	suppress-genoffs 
2020-12-06T16:35:46.8880538Z 	suppress-gensyms 
2020-12-06T16:35:46.8880917Z 	wincrt 
2020-12-06T16:35:46.8881256Z 	mallocMTd 
2020-12-06T16:35:46.8881920Z 	mallocMD 
2020-12-06T16:35:46.8882339Z 	cs2bugMTd 
2020-12-06T16:35:46.8882725Z 	cs2bugMTdZI 
2020-12-06T16:35:46.8883101Z 	cs2bugMD 
2020-12-06T16:35:46.8883512Z 	operatorsMDd 
2020-12-06T16:35:46.8883912Z 	cs2bugMDd 
2020-12-06T16:35:46.8884279Z 	gdi 
2020-12-06T16:35:46.8884610Z 	syscalls_win 
2020-12-06T16:35:46.8884974Z 	handle 
2020-12-06T16:35:46.8885295Z 	handle_only 
2020-12-06T16:35:46.8885765Z 	blacklist 
2020-12-06T16:35:46.8886078Z 	memlayout 
2020-12-06T16:35:46.8886388Z 	nudge 
2020-12-06T16:35:46.8886703Z 	nosyms 
2020-12-06T16:35:46.8886992Z 	syscall_file_all 
2020-12-06T16:35:46.8887388Z 	(ignore: i#1938) 	app_suite.pattern 
2020-12-06T16:35:46.8887792Z 	syscall_file_gen 
2020-12-06T16:35:46.8888143Z 	fuzz_buffer.cpp 
2020-12-06T16:35:46.8888612Z 	fuzz_buffer.cpp.demangled 
2020-12-06T16:35:46.8889061Z 	drstrace_unit_tests
This was referenced Dec 7, 2020
Closed
Merged
@derekbruening
Copy link
Contributor Author

Most failures are due to extra uninits or other error reports, likely false positives.

RtlCaptureContext is frequent: #2333.
Adding to default suppressions => 6 tests now pass: down to 33 failures.
These 6 now pass: malloc, noreplace_realloc, redzone16, redzone1024, mallocMTd, mallocMD.
I assume others are now closer to passing as well b/c there were enough instances (48)
for many tests.

derekbruening added a commit that referenced this issue Dec 7, 2020
@derekbruening
Print contents of results file on CI test failures
66e01eb 
On a failure to match either the stderr summary or individual results,
if the env var CI is set (true for Github Actions), runtest.cmake now
prints out the contents of the results file(s).  This greatly aids
remote debugging of CI failures.

Issue: #2334
@derekbruening derekbruening mentioned this issue Dec 7, 2020
Merged
derekbruening added a commit that referenced this issue Dec 7, 2020
@derekbruening
Print contents of results file on CI test failures (#2335)
9a88999 
On a failure to match either the stderr summary or individual results,
if the env var CI is set (true for Github Actions), runtest.cmake now
prints out the contents of the results file(s).  This greatly aids
remote debugging of CI failures.

Issue: #2334
@derekbruening derekbruening mentioned this issue Dec 11, 2020
Closed
derekbruening added a commit that referenced this issue Dec 12, 2020
@derekbruening
i#2334: Set up Windows CI on Github Actions (#2326)
6cb4599 
Sets up 3 jobs on Windows Server 2016 on Github Actions:
+ 32-bit debug build and tests
+ 64-bit debug build and tests
+ 32-bit and 64-bit release build plus drheapstat debug build, no tests

Adds parameters to runsuite.cmake for the control split into these 3 jobs.
Each job is roughly under 15 minutes.
Each job downloads and installs ninja, doxygen, and WiX and uses VS2017.

Adapts runsuite_wrapper.pl to use native Windows perl rather than Cygwin perl.  No fork is available, so we tee the output to a file and read the file back in.
Fixes a missed final line in the suite results processing.

Adds default suppressions for invalid heap arguments (#2339) and leaks (#2340) in the VS2017 CRT.

Generalizes test output for cs2bug and the suppress* tests in an attempt to get them to pass.

Augments umbra_address_space_init() with a better error message to avoid cases like #2328 in the future.

Shrinks the timeout to 60s for the failing umbra_client_faulty_redzone to eliminate several minutes of testing time just waiting for that test to time out; #2341 covers fixing it.

Augments the test ignore list in order to get the jobs green.

Issue: #2328, #2334, #2339, #2340, #2341
@derekbruening derekbruening mentioned this issue Dec 12, 2020
Closed
@derekbruening
Copy link
Contributor Author

The main PR #2326 is now in after a lot of work.
Package building is not there yet but that's part of #2323.
The only thing left here is to remove the Appveyor config and trigger.

derekbruening added a commit that referenced this issue Dec 12, 2020
@derekbruening
i#2334 Win GA: Remove appveyor config
dcebf70 
Removes the .appveyor.yml file as we have switched to Github Actions,
which offers parallel jobs and auto-cancellation.

Fixes #2334
@derekbruening derekbruening mentioned this issue Dec 12, 2020
Merged
derekbruening added a commit that referenced this issue Dec 12, 2020
@derekbruening
i#2334 Win GA: Remove appveyor config (#2346)
adc18f4 
Removes the .appveyor.yml file as we have switched to Github Actions,
which offers parallel jobs and auto-cancellation.

Fixes #2334
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component-Tests OpSys-Windows
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

i#2334 Win GA: Remove appveyor config DynamoRIO/drmemory
1 participant
@derekbruening