Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix failing MacOS tests #1815

Open
derekbruening opened this issue Oct 16, 2015 · 2 comments
Open

fix failing MacOS tests #1815

derekbruening opened this issue Oct 16, 2015 · 2 comments
Labels
Bug-Assert Component-Tests good first issue help wanted OpSys-OSX

Comments

@derekbruening
Copy link
Contributor

Split from #58.

Only recently did we get the test suite to build on Mac (we were blocked by #1409). Now they all build but only a quarter pass:

> egrep '^Test Pass|Testing:' ~/derek/dr/git/build_x86_dbg_tests/Testing/Temporary/LastTest.log | grep -B 1 Pass | grep '^[0-9]' | sort -n 
1/184 Testing: unit_tests
2/184 Testing: samples_proj
3/184 Testing: code_api|common.broadfun
4/184 Testing: code_api|common.fib
7/184 Testing: code_api|common.floatpc
8/184 Testing: code_api|common.floatpc_xl8all
9/184 Testing: code_api|common.getretaddr
11/184 Testing: code_api|libutil.frontend_test
16/184 Testing: code_api|linux.exit
20/184 Testing: code_api|linux.infinite
23/184 Testing: code_api|linux.signal0000
24/184 Testing: code_api|linux.signal0001
25/184 Testing: code_api|linux.signal0010
27/184 Testing: code_api|linux.signal0100
28/184 Testing: code_api|linux.signal0101
29/184 Testing: code_api|linux.signal0110
47/184 Testing: code_api|linux.sigaction
53/184 Testing: code_api|linux.app_tls
55/184 Testing: code_api|linux.fib-conflict
57/184 Testing: code_api|linux.fib-pie
69/184 Testing: code_api|security-common.codemod
71/184 Testing: code_api|security-common.retexisting
72/184 Testing: code_api|security-common.ret_noncall_trace
79/184 Testing: code_api|client.abort
80/184 Testing: code_api|client.crashmsg
84/184 Testing: code_api|client.count-ctis-noopt
88/184 Testing: code_api|client.exception
104/184 Testing: code_api|client.timer
105/184 Testing: code_api|client.syscall-mod
107/184 Testing: code_api|client.cbr-retarget
109/184 Testing: code_api|client.flush
112/184 Testing: code_api|client.truncate
113/184 Testing: code_api|client.dr_options
114/184 Testing: code_api|client.unregister
116/184 Testing: code_api|client.null_instrument
118/184 Testing: code_api|client.option_parse
119/184 Testing: code_api|client.drcontainers-test
129/184 Testing: code_api|client.fcache_shift
132/184 Testing: code_api|client.destructor
150/184 Testing: code_api|sample.signal
163/184 Testing: code_api|tool.drcpusim.simple
167/184 Testing: code_api|tool.drcpusim.cpuid-Banias
168/184 Testing: code_api|tool.drcpusim.cpuid-Prescott
169/184 Testing: code_api|tool.drcpusim.cpuid-Presler
170/184 Testing: code_api|tool.drcpusim.cpuid-Merom
171/184 Testing: code_api|tool.drcpusim.cpuid-Penryn
172/184 Testing: code_api|tool.drcpusim.cpuid-Westmere
173/184 Testing: code_api|tool.drcpusim.cpuid-Nehalem
174/184 Testing: code_api|api.ir
175/184 Testing: code_api|api.dis
177/184 Testing: code_api|api.ir-static
178/184 Testing: code_api|api.static

28% tests passed, 132 tests failed out of 184

The following tests FAILED:
          5 - code_api|common.decode-bad (Failed)
          6 - code_api|common.eflags (Failed)
         10 - code_api|common.segfault (Failed)
         12 - code_api|linux.execve-null (Failed)
         13 - code_api|linux.child-blacklist (Failed)
         14 - code_api|linux.child-whitelist (Failed)
         15 - code_api|linux.execve-rec (Failed)
         17 - code_api|linux.fork (Failed)
         18 - code_api|linux.fork-sleep (Failed)
         19 - code_api|linux.fork-MSR (Failed)
         21 - code_api|linux.longjmp (Failed)
         22 - code_api|linux.mmap (Failed)
         26 - code_api|linux.signal0011 (Failed)
         30 - code_api|linux.signal0111 (Failed)
         31 - code_api|linux.signal1000 (Failed)
         32 - code_api|linux.signal1001 (Failed)
         33 - code_api|linux.signal1010 (Failed)
         34 - code_api|linux.signal1011 (Failed)
         35 - code_api|linux.signal1100 (Failed)
         36 - code_api|linux.signal1101 (Failed)
         37 - code_api|linux.signal1110 (Failed)
         38 - code_api|linux.signal1111 (Failed)
         39 - code_api|linux.sigplain000 (Failed)
         40 - code_api|linux.sigplain001 (Failed)
         41 - code_api|linux.sigplain010 (Failed)
         42 - code_api|linux.sigplain011 (Failed)
         43 - code_api|linux.sigplain100 (Failed)
         44 - code_api|linux.sigplain101 (Failed)
         45 - code_api|linux.sigplain110 (Failed)
         46 - code_api|linux.sigplain111 (Failed)
         48 - code_api|linux.alarm (Failed)
         49 - code_api|linux.eintr (Failed)
         50 - code_api|linux.eintr-noinline (Failed)
         51 - code_api|linux.sigsuspend (Failed)
         52 - code_api|linux.signest (Failed)
         54 - code_api|linux.readlink (Failed)
         56 - code_api|linux.fib-conflict-early (Failed)
         58 - code_api|linux.vfork (Failed)
         59 - code_api|linux.thread-reset (OTHER_FAULT)
         60 - code_api|linux.clone-reset (OTHER_FAULT)
         61 - code_api|pthreads.pthreads (Failed)
         62 - code_api|pthreads.pthreads_exit (Failed)
         63 - code_api|pthreads.ptsig_FLAKY (Failed)
         64 - code_api|pthreads.pthreads_fork_FLAKY (Failed)
         65 - code_api|linux.reset (Failed)
         66 - code_api|linux.freeze_FLAKY (Failed)
         67 - code_api|linux.persist_FLAKY (Failed)
         68 - code_api|linux.persist-use_FLAKY (Failed)
         70 - code_api|security-common.decode-bad-stack (Failed)
         73 - code_api|security-common.retnonexisting (Failed)
         74 - code_api|security-common.selfmod2 (Failed)
         75 - code_api|security-common.selfmod (Failed)
         76 - code_api|security-common.TestAllocWE (Failed)
         77 - code_api|security-common.TestMemProtChg_FLAKY (Failed)
         78 - code_api|client.call-retarget (Failed)
         81 - code_api|client.alloc (Failed)
         82 - code_api|client.cleancall (Failed)
         83 - code_api|client.count-ctis (Failed)
         85 - code_api|client.syscall (Failed)
         86 - code_api|client.count-bbs (Failed)
         87 - code_api|client.app_inscount (Failed)
         89 - code_api|client.partial_module_map (Failed)
         90 - code_api|client.segfault (Failed)
         91 - code_api|client.execfault (Failed)
         92 - code_api|client.vg-annot (Failed)
         93 - code_api|client.vg-annot.full-decode (Failed)
         94 - code_api|client.vg-annot.full-decode.tiny-bb (Failed)
         95 - code_api|client.vg-annot.bb-truncate-1 (Failed)
         96 - code_api|client.vg-annot.bb-truncate-2 (Failed)
         97 - code_api|client.vg-annot-opt (Failed)
         98 - code_api|client.vg-annot-opt.full-decode (Failed)
         99 - code_api|client.vg-annot-opt.full-decode.tiny-bb (Failed)
        100 - code_api|client.vg-annot-opt.bb-truncate (Failed)
        101 - code_api|client.events (Failed)
        102 - code_api|client.events_cpp (Failed)
        103 - code_api|client.nudge_test (Failed)
        106 - code_api|client.signal (Failed)
        108 - code_api|client.file_io (Failed)
        110 - code_api|client.thread (Failed)
        111 - code_api|client.strace (Failed)
        115 - code_api|client.inline (Failed)
        117 - code_api|client.large_options (Failed)
        120 - code_api|client.drmgr-test (Failed)
        121 - code_api|client.drreg-test (Failed)
        122 - code_api|client.drx-test (Failed)
        123 - code_api|client.drwrap-test (Failed)
        124 - code_api|client.drwrap-test-callconv (Failed)
        125 - code_api|client.drsyms-test (Failed)
        126 - code_api|client.drutil-test (Failed)
        127 - code_api|client.pcache (Failed)
        128 - code_api|client.pcache-use (Failed)
        130 - code_api|client.nudge_ex (Failed)
        131 - code_api|client.retaddr (Failed)
        133 - code_api|sample.bbbuf (Failed)
        134 - code_api|sample.bbsize (Failed)
        135 - code_api|sample.empty (Failed)
        136 - code_api|sample.memtrace_simple (Failed)
        137 - code_api|sample.bbcount (Failed)
        138 - code_api|sample.cbr (Failed)
        139 - code_api|sample.countcalls (Failed)
        140 - code_api|sample.div (Failed)
        141 - code_api|sample.inc2add (Failed)
        142 - code_api|sample.memtrace_x86 (Failed)
        143 - code_api|sample.prefetch (Failed)
        144 - code_api|sample.ssljack (Failed)
        145 - code_api|sample.modxfer (Failed)
        146 - code_api|sample.modxfer_app2lib (Failed)
        147 - code_api|sample.instrcalls (Failed)
        148 - code_api|sample.cbrtrace (Failed)
        149 - code_api|sample.wrap (Failed)
        151 - code_api|sample.syscall (Failed)
        152 - code_api|sample.inline (Failed)
        153 - code_api|sample.inscount (Failed)
        154 - code_api|sample.inscount.cleancall (Failed)
        155 - code_api|sample.opcodes (Failed)
        156 - code_api|sample.stl_test (Failed)
        157 - code_api|tool.drltrace (Failed)
        158 - code_api|tool.drcov.fib (Failed)
        159 - code_api|tool.drcachesim.simple (Failed)
        160 - code_api|tool.drcachesim.TLB-simple (Failed)
        161 - code_api|tool.drcachesim.phys (Failed)
        162 - code_api|tool.drcachesim.multiproc (Failed)
        164 - code_api|tool.drcpusim.cpuid-Klamath (Failed)
        165 - code_api|tool.drcpusim.cpuid-Deschutes (Failed)
        166 - code_api|tool.drcpusim.cpuid-Pentium3 (Failed)
        176 - code_api|api.startstop_FLAKY (Failed)
        179 - code_api,opt_memory|client.events (Failed)
        180 - code_api,opt_speed|client.events (Failed)
        181 - code_api,thread_private|client.events (Failed)
        182 - code_api,disable_traces|client.events (Failed)
        183 - code_api,thread_private,disable_traces|client.events (Failed)
        184 - code_api,no_early_inject|client.events (Failed)
Errors while running CTest
@derekbruening
Copy link
Contributor Author

These are now failing on 64-bit:

debug-internal-64: 45 tests passed, **** 4 tests failed, but ignoring 0 for i: ****
	code_api|common.decode-bad 
	code_api|client.abort 
	code_api|client.timer 
	code_api|tool.drcpusim.simple 

Privileged instructions about to happen
<Application /Users/travis/build/DynamoRIO/dynamorio/build_debug-internal-64/suite/tests/bin/common.decode-bad (33461).  Internal Error: DynamoRIO debug check failure: /Users/travis/build/DynamoRIO/dynamorio/core/unix/signal.c:2461 mc != NULL && sc != NULL
Error occurred @1408 frags)

  /Users/travis/build/DynamoRIO/dynamorio/build_debug-internal-64/bin64/drrun;-s;90;-quiet;-debug;-killpg;-stderr_mask;0xC;-dumpcore_mask;0;-code_api;-c;/Users/travis/build/DynamoRIO/dynamorio/build_debug-internal-64/suite/tests/bin/libclient.abort.dll.dylib;--;/Users/travis/build/DynamoRIO/dynamorio/build_debug-internal-64/suite/tests/bin/client.abort
  failed: 255 != 8 

<Application /Users/travis/build/DynamoRIO/dynamorio/build_debug-internal-64/suite/tests/bin/client.timer (33895).  Internal Error: DynamoRIO debug check failure: /Users/travis/build/DynamoRIO/dynamorio/core/unix/signal.c:2461 mc != NULL && sc != NULL
(Error occurred @1407 frags)

38/49 Test #216: code_api|tool.drcpusim.simple ................***Failed  Required regular expression not found.Regex=[^Hello, world!
$
]  0.36 sec
<Invalid Westmere instruction "xsave" @ libdyld.dylib+0x1281.  Aborting.>

As part of #1979 I'm removing them from the list for now.

@derekbruening derekbruening mentioned this issue Feb 13, 2020
Open
derekbruening added a commit that referenced this issue Feb 13, 2020
@derekbruening
i#1979 Mac64: Update Travis to test 64-bit Mac and not 32-bit (#4095)
2d699bc 
We no longer support 32-bit Mac as it does not really exist anymore.
We switch Travis to 64-bit-only and disable a few failing tests which i#1815
covers fixing.

Issue: #1979, #1815
@derekbruening
Copy link
Contributor Author

derekbruening commented Feb 14, 2020 •
edited
Loading

On OSX 10.14 (10.13 was the Travis default):

https://travis-ci.com/DynamoRIO/dynamorio/jobs/287117155

49913/45 Test  #52: code_api|linux.sigaction .....................***Failed  Required regular expression not found. Regex=[^Sending SIGTERM first time
500Sending SIGTERM second time
501$
502]  0.41 sec
503Assertion failed: (rc == 0 && old_act.sa_sigaction == first_act.sa_sigaction && *(long *)&old_act.sa_mask == *(long *)&first_act.sa_mask), function test_query, file /Users/travis/build/DynamoRIO/dynamorio/suite/tests/linux/sigaction.c, line 83.
504<Application /Users/travis/build/DynamoRIO/dynamorio/build_debug-internal-64/suite/tests/bin/linux.sigaction (23090).  Internal Error: DynamoRIO debug check failure: /Users/travis/build/DynamoRIO/dynamorio/core/unix/os.c:1801 false
505(Error occurred @1426 frags)

derekbruening added a commit that referenced this issue Feb 14, 2020
@derekbruening
Remove sigaction until fixed by #1815
29f839c
derekbruening added a commit that referenced this issue Feb 14, 2020
@derekbruening
i#1979 Mac64: Move Travis to OSX 10.14 (#4096)
f476716 
Moves Travis from OSX 10.13 to 10.14.
Removes the sigaction test which fails there (fixing is under #1815).

Issue: #1979, #1815
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug-Assert Component-Tests good first issue help wanted OpSys-OSX
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@derekbruening