Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#i2016 slowpath-aarch64-port inital #2411

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

#i2016 slowpath-aarch64-port inital #2411

wants to merge 8 commits into from

Conversation

gregcawthorne
Copy link
Contributor

AArch64 port of drmemory.

Only contains slowpath support with shared_slowpath off.

Pattern mode and fastpath modes are being worked on separately.

Currently this build does break some x86 functionality.

Depends on:
https://github.com/DynamoRIO/dynamorio/tree/mem-ref-for-clean-calls-aarch64/core

Current tests we have analysed:
Test project /home/grecaw01/APD-testing/drmem-upstream3/drmemory/build
Start 1: drmf_proj
1/49 Test #1: drmf_proj ......................... Passed 0.45 sec
Start 2: unit_tests
2/49 Test #2: unit_tests ........................ Passed 0.02 sec
Start 3: hello
3/49 Test #3: hello ............................. Passed 3.55 sec
Start 4: free
4/49 Test #4: free .............................. Passed 3.67 sec
Start 5: malloc
5/49 Test #5: malloc ............................ Passed 3.88 sec
Start 6: leak_indirect
6/49 Test #6: leak_indirect ..................... Passed 3.52 sec
Start 7: patterns
7/49 Test #7: patterns .......................... Passed 3.93 sec
Start 8: free.exitcode
8/49 Test #8: free.exitcode ..................... Passed 3.64 sec
Start 9: track_origins
9/49 Test #9: track_origins .....................***Failed 0.34 sec
Start 10: free.pattern
10/49 Test #10: free.pattern ......................***Failed 0.35 sec
Start 11: malloc.pattern
11/49 Test #11: malloc.pattern ....................***Failed 0.34 sec
Start 12: track_origins.pattern
12/49 Test #12: track_origins.pattern .............***Failed 0.34 sec
Start 13: fuzz_corpus
13/49 Test #13: fuzz_corpus ....................... Passed 3.56 sec
Start 14: fuzz_buffer
14/49 Test #14: fuzz_buffer ....................... Passed 4.62 sec
Start 15: fuzz_buffer.replace_buffer
15/49 Test #15: fuzz_buffer.replace_buffer ........ Passed 4.62 sec
Start 16: fuzz_buffer.overflow
16/49 Test #16: fuzz_buffer.overflow ..............***Failed 0.34 sec
Start 17: fuzz_buffer.mutator.o-b-s-3
17/49 Test #17: fuzz_buffer.mutator.o-b-s-3 ....... Passed 4.59 sec
Start 18: fuzz_buffer.mutator.r-b-s-3
18/49 Test #18: fuzz_buffer.mutator.r-b-s-3 ....... Passed 4.63 sec
Start 19: fuzz_buffer.mutator.o-b-3
19/49 Test #19: fuzz_buffer.mutator.o-b-3 ......... Passed 4.60 sec
Start 20: fuzz_buffer.mutator.r-n
20/49 Test #20: fuzz_buffer.mutator.r-n ........... Passed 4.54 sec
Start 21: fuzz_buffer.mutator.random_seed
21/49 Test #21: fuzz_buffer.mutator.random_seed ... Passed 4.57 sec
Start 22: fuzz_buffer.one-input
22/49 Test #22: fuzz_buffer.one-input ............. Passed 3.82 sec
Start 23: fuzz_buffer.load_input
23/49 Test #23: fuzz_buffer.load_input ............ Passed 3.81 sec
Start 24: fuzz_buffer.skip_initial
24/49 Test #24: fuzz_buffer.skip_initial .......... Passed 4.01 sec
Start 25: fuzz_buffer.fixed_size
25/49 Test #25: fuzz_buffer.fixed_size ............ Passed 5.36 sec
Start 26: fuzz_buffer.offset
26/49 Test #26: fuzz_buffer.offset ................ Passed 5.42 sec
Start 27: fuzz_buffer.module_name
27/49 Test #27: fuzz_buffer.module_name ........... Passed 4.58 sec
Start 28: fuzz_buffer.dictionary
28/49 Test #28: fuzz_buffer.dictionary ............ Passed 4.20 sec
Start 29: fuzz_buffer.cpp
29/49 Test #29: fuzz_buffer.cpp ................... Passed 17.77 sec
Start 30: fuzz_custom_mutator
30/49 Test #30: fuzz_custom_mutator ............... Passed 4.57 sec
Start 31: drsyscall_test
31/49 Test #31: drsyscall_test .................... Passed 0.22 sec
Start 32: strace_test
32/49 Test #32: strace_test ....................... Passed 0.22 sec
Start 33: drfuzz_test_empty
33/49 Test #33: drfuzz_test_empty ................. Passed 0.22 sec
Start 34: drfuzz_test_mutator
34/49 Test #34: drfuzz_test_mutator ............... Passed 2.38 sec
Start 35: drfuzz_test_repeat
35/49 Test #35: drfuzz_test_repeat ................***Failed
Start 36: drfuzz_test_segfault
36/49 Test #36: drfuzz_test_segfault .............. Passed 0.20 sec
Start 37: drfuzz_test_app_abort
37/49 Test #37: drfuzz_test_app_abort ............. Passed 0.22 sec
Start 38: drfuzz_test_no_crash
38/49 Test #38: drfuzz_test_no_crash .............. Passed 0.22 sec
Start 39: umbra_test_empty
39/49 Test #39: umbra_test_empty .................. Passed 0.22 sec
Start 40: umbra_test_overlap
40/49 Test #40: umbra_test_overlap ................ Passed 0.23 sec
Start 41: umbra_test_shadow_mem
41/49 Test #41: umbra_test_shadow_mem ............. Passed 0.30 sec
Start 42: umbra_test_insert_app_to_shadow
42/49 Test #42: umbra_test_insert_app_to_shadow ... Passed 0.29 sec
Start 43: umbra_test_consistency
43/49 Test #43: umbra_test_consistency ............ Passed 0.30 sec
Start 44: umbra_test_allscales
44/49 Test #44: umbra_test_allscales .............. Passed 0.39 sec
Start 45: drltrace
45/49 Test #45: drltrace .......................... Passed 0.35 sec
Start 46: drltrace_libcalls
46/49 Test #46: drltrace_libcalls ................. Passed 0.36 sec
Start 47: drltrace_symargs
47/49 Test #47: drltrace_symargs .................. Passed 0.36 sec
Start 48: drltrace_libargs
48/49 Test #48: drltrace_libargs .................. Passed 0.35 sec
Start 49: strace_sample
49/49 Test #49: strace_sample ..................... Passed 0.22 sec

88% tests passed, 6 tests failed out of 49

@gregcawthorne
slowpath-aarch64-port inital
43fca9e 
AArch64 port of drmemory.

Only contains slowpath support with shared_slowpath off.

Pattern mode and fastpath modes are being worked on separately.

Currently this build does break some x86 functionality.

Depends on:
https://github.com/DynamoRIO/dynamorio/tree/mem-ref-for-clean-calls-aarch64/core

Current tests we have analysed:
Test project /home/grecaw01/APD-testing/drmem-upstream3/drmemory/build
      Start  1: drmf_proj
 1/49 Test  #1: drmf_proj .........................   Passed    0.45 sec
      Start  2: unit_tests
 2/49 Test  #2: unit_tests ........................   Passed    0.02 sec
      Start  3: hello
 3/49 Test  #3: hello .............................   Passed    3.55 sec
      Start  4: free
 4/49 Test  #4: free ..............................   Passed    3.67 sec
      Start  5: malloc
 5/49 Test  #5: malloc ............................   Passed    3.88 sec
      Start  6: leak_indirect
 6/49 Test  #6: leak_indirect .....................   Passed    3.52 sec
      Start  7: patterns
 7/49 Test  #7: patterns ..........................   Passed    3.93 sec
      Start  8: free.exitcode
 8/49 Test  #8: free.exitcode .....................   Passed    3.64 sec
      Start  9: track_origins
 9/49 Test  #9: track_origins .....................***Failed    0.34 sec
      Start 10: free.pattern
10/49 Test #10: free.pattern ......................***Failed    0.35 sec
      Start 11: malloc.pattern
11/49 Test #11: malloc.pattern ....................***Failed    0.34 sec
      Start 12: track_origins.pattern
12/49 Test #12: track_origins.pattern .............***Failed    0.34 sec
      Start 13: fuzz_corpus
13/49 Test #13: fuzz_corpus .......................   Passed    3.56 sec
      Start 14: fuzz_buffer
14/49 Test #14: fuzz_buffer .......................   Passed    4.62 sec
      Start 15: fuzz_buffer.replace_buffer
15/49 Test #15: fuzz_buffer.replace_buffer ........   Passed    4.62 sec
      Start 16: fuzz_buffer.overflow
16/49 Test #16: fuzz_buffer.overflow ..............***Failed    0.34 sec
      Start 17: fuzz_buffer.mutator.o-b-s-3
17/49 Test #17: fuzz_buffer.mutator.o-b-s-3 .......   Passed    4.59 sec
      Start 18: fuzz_buffer.mutator.r-b-s-3
18/49 Test #18: fuzz_buffer.mutator.r-b-s-3 .......   Passed    4.63 sec
      Start 19: fuzz_buffer.mutator.o-b-3
19/49 Test #19: fuzz_buffer.mutator.o-b-3 .........   Passed    4.60 sec
      Start 20: fuzz_buffer.mutator.r-n
20/49 Test #20: fuzz_buffer.mutator.r-n ...........   Passed    4.54 sec
      Start 21: fuzz_buffer.mutator.random_seed
21/49 Test #21: fuzz_buffer.mutator.random_seed ...   Passed    4.57 sec
      Start 22: fuzz_buffer.one-input
22/49 Test #22: fuzz_buffer.one-input .............   Passed    3.82 sec
      Start 23: fuzz_buffer.load_input
23/49 Test #23: fuzz_buffer.load_input ............   Passed    3.81 sec
      Start 24: fuzz_buffer.skip_initial
24/49 Test #24: fuzz_buffer.skip_initial ..........   Passed    4.01 sec
      Start 25: fuzz_buffer.fixed_size
25/49 Test #25: fuzz_buffer.fixed_size ............   Passed    5.36 sec
      Start 26: fuzz_buffer.offset
26/49 Test #26: fuzz_buffer.offset ................   Passed    5.42 sec
      Start 27: fuzz_buffer.module_name
27/49 Test #27: fuzz_buffer.module_name ...........   Passed    4.58 sec
      Start 28: fuzz_buffer.dictionary
28/49 Test #28: fuzz_buffer.dictionary ............   Passed    4.20 sec
      Start 29: fuzz_buffer.cpp
29/49 Test #29: fuzz_buffer.cpp ...................   Passed   17.77 sec
      Start 30: fuzz_custom_mutator
30/49 Test #30: fuzz_custom_mutator ...............   Passed    4.57 sec
      Start 31: drsyscall_test
31/49 Test #31: drsyscall_test ....................   Passed    0.22 sec
      Start 32: strace_test
32/49 Test #32: strace_test .......................   Passed    0.22 sec
      Start 33: drfuzz_test_empty
33/49 Test #33: drfuzz_test_empty .................   Passed    0.22 sec
      Start 34: drfuzz_test_mutator
34/49 Test #34: drfuzz_test_mutator ...............   Passed    2.38 sec
      Start 35: drfuzz_test_repeat
35/49 Test #35: drfuzz_test_repeat ................***Failed
      Start 36: drfuzz_test_segfault
36/49 Test #36: drfuzz_test_segfault ..............   Passed    0.20 sec
      Start 37: drfuzz_test_app_abort
37/49 Test #37: drfuzz_test_app_abort .............   Passed    0.22 sec
      Start 38: drfuzz_test_no_crash
38/49 Test #38: drfuzz_test_no_crash ..............   Passed    0.22 sec
      Start 39: umbra_test_empty
39/49 Test #39: umbra_test_empty ..................   Passed    0.22 sec
      Start 40: umbra_test_overlap
40/49 Test #40: umbra_test_overlap ................   Passed    0.23 sec
      Start 41: umbra_test_shadow_mem
41/49 Test #41: umbra_test_shadow_mem .............   Passed    0.30 sec
      Start 42: umbra_test_insert_app_to_shadow
42/49 Test #42: umbra_test_insert_app_to_shadow ...   Passed    0.29 sec
      Start 43: umbra_test_consistency
43/49 Test #43: umbra_test_consistency ............   Passed    0.30 sec
      Start 44: umbra_test_allscales
44/49 Test #44: umbra_test_allscales ..............   Passed    0.39 sec
      Start 45: drltrace
45/49 Test #45: drltrace ..........................   Passed    0.35 sec
      Start 46: drltrace_libcalls
46/49 Test #46: drltrace_libcalls .................   Passed    0.36 sec
      Start 47: drltrace_symargs
47/49 Test #47: drltrace_symargs ..................   Passed    0.36 sec
      Start 48: drltrace_libargs
48/49 Test #48: drltrace_libargs ..................   Passed    0.35 sec
      Start 49: strace_sample
49/49 Test #49: strace_sample .....................   Passed    0.22 sec

88% tests passed, 6 tests failed out of 49
@derekbruening
Copy link
Contributor

Hi @gregcawthorne -- wondering if you could provide a status update on where things are with the aarch64 port? It sounds like you have the slowpath working for small apps locally?

@gregcawthorne
Copy link
Contributor Author

Hi yes the full slowpath port works with small apps locally.

I'd say its definitely a useful contribution, but there are of course some test cases that still fail and the fast path to do.

Greg

@derekbruening
Copy link
Contributor

Hi yes the full slowpath port works with small apps locally.

I'd say its definitely a useful contribution, but there are of course some test cases that still fail and the fast path to do.

+1, definitely useful, and have to start somewhere. There are some key use cases of the tool on things like computer science class homeworks where working on simple apps, even if not super fast, is sufficient.

@derekbruening
Copy link
Contributor

There is some interest in AArch64 Dr. Memory, even if only the slowpath works, for testing student homework submissions. @gregcawthorne -- would it be possible to get your code pushed upstream soon?

@gregcawthorne
Copy link
Contributor Author

Hi Derek.

Are you happy with the style of me choosing different sub directories and making PRs for them one by one?

@derekbruening
Copy link
Contributor

derekbruening commented Aug 23, 2021 via email

@derekbruening
Copy link
Contributor

@gregcawthorne -- we would like to get AArch64 Dr. Memory going (slowpath only is fine since only running small apps) for a CS class for student homeworks.

@derekbruening
Copy link
Contributor

@gregcawthorne Are you using drreg in your AArch64 port?

@gregcawthorne
Copy link
Contributor Author

gregcawthorne commented Sep 22, 2021 via email

@derekbruening
Copy link
Contributor

Yes I am making use of drreg in various places.

Does this mean you have merged in the branch https://github.com/DynamoRIO/drmemory/tree/i1795-use-drreg, or you have a completely parallel use of drreg to the x86/ARM use of the local register handling code?

@derekbruening derekbruening reopened this Aug 28, 2022
@gregcawthorne
Copy link
Contributor Author

Yes I am making use of drreg in various places.

Does this mean you have merged in the branch https://github.com/DynamoRIO/drmemory/tree/i1795-use-drreg, or you have a completely parallel use of drreg to the x86/ARM use of the local register handling code?

Hi, sorry I didn't get back to you on this.
I was using drreg in my own way, but I might have used https://github.com/DynamoRIO/drmemory/tree/i1795-use-drreg, for some inspiration.

I also had some very rough work on fastpath port I have sent to Yury, Assad and Al Grant. That also uses drreg quite a bit.

@derekbruening
Copy link
Contributor

As the errors show this does not build on any platform, let alone aarch64. Looks like there is a bunch of work to get even the slowpath building and working.

@derekbruening
Copy link
Contributor

There are a number of uses in this code of things that are missing from dynamorio. It's as though there was a private branch of dynamorio adding various features used in this drmemory branch but it was never committed. Things like:

  • INSTR_CREATE_lsl in umbra_64.c
  • OP_udf in stack.c
  • Implementing reg_get_value_ex()

There were past requests to please upstream local features -- it sounds like it hasn't happened and that work was lost??

derekbruening and others added 5 commits September 24, 2022 11:18
@derekbruening
Add missing pieces and fix build errors, including:
c6f0042 
+ Add missing get_unwind_registers()
+ Work around missing OP_udf with raw instr
+ Work around missing INSTR_CREATE_lsl with OP_ubfm
+ Work around missing reg_get_value_ex() with reg_get_value() and GPR assert
+ Fix ARM build breakage
+ Fix style violations
@derekbruening
Support gcc 12
8bcbc7d 
Fixes build warnings with gcc 12.
Updates DR to 44312ad1c to fix gcc 12 issues there.
@derekbruening
Merge branch 'master' of github.com:DynamoRIO/drmemory into slowpath-…
cc56f03 
…aarch64-port
@derekbruening
Bump mac supported version b/c we did update GA CI to 11
2f81f11
@AssadHashmi
i#2016 AArch64: Fix build errors
27ec864 
Updates to get drmemory working again on AArch64 after changes to
dynamorio code over the past ~2 years, since September 2022.

Changes due to:
- Symbol conflicts with IN, OUT and INOUT: i#3348, PR #6455
- Namespaces added to droption: i#4343, PR#6193

Also updated dates in copyright header comments for all files changed.

Status of unit tests:

82% tests passed, 9 tests failed out of 50

Total Test time (real) =  71.44 sec

The following tests FAILED:
          9 - track_origins (Failed)
         10 - free.pattern (Failed)
         11 - malloc.pattern (Failed)
         12 - track_origins.pattern (Failed)
         16 - fuzz_buffer.overflow (Failed)
         35 - drfuzz_test_repeat (Failed)
         42 - umbra_test_insert_app_to_shadow (Failed)
         43 - umbra_wild_shadow (Failed)
         48 - drltrace_symargs (Failed)

Basic binaries run to completion, e.g.

$ drmemory -- /usr/bin/pwd
~Dr.M~~ Dr. Memory version 2.5.19259
~Dr.M~~ ERROR: Failed to find "main" for limiting memory dump
~Dr.M~~ WARNING: application is missing line number information.
. . .
. . .
~Dr.M~~ ERRORS FOUND:
~Dr.M~~       0 unique,     0 total unaddressable access(es)
~Dr.M~~      79 unique,   786 total uninitialized access(es)
~Dr.M~~       0 unique,     0 total invalid heap argument(s)
~Dr.M~~       0 unique,     0 total warning(s)
~Dr.M~~       6 unique,    52 total,   1862 byte(s) of leak(s)
~Dr.M~~       0 unique,     0 total,      0 byte(s) of possible leak(s)
~Dr.M~~ ERRORS IGNORED:
~Dr.M~~      27 unique,   109 total,  17898 byte(s) of still-reachable allocation(s)
~Dr.M~~          (re-run with "-show_reachable" for details)
~Dr.M~~ Details: /path/to/logs/DrMemory-pwd.3313045.000/results.txt
$
@AssadHashmi
Copy link

@derekbruening I pushed a patch last week to fix build errors but none of the checks/tests seem to be running.

@derekbruening
Copy link
Contributor

@derekbruening I pushed a patch last week to fix build errors but none of the checks/tests seem to be running.

Not sure why they didn't trigger: they ran for other PR's a few months ago. There are conflicts: you'd think they would only block merging, but it does say "Use the command line to resolve conflicts before continuing." Did you try searching the github docs about this? A quick search finds https://github.com/orgs/community/discussions/26304 which implies that actions are not triggered if there's a conflict: so that is likely the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gregcawthorne @derekbruening @AssadHashmi