-
Notifications
You must be signed in to change notification settings - Fork 567
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BinExport2 backend #1950
Merged
Merged
BinExport2 backend #1950
Changes from 221 commits
Commits
Show all changes
252 commits
Select commit
Hold shift + click to select a range
ad732fc
elf: os: detect Android via clang compiler .ident note
williballenthin 270956b
elf: os: detect Android via dependency on liblog.so
williballenthin e064ce8
main: split main into a bunch of "main routines"
williballenthin 30fe988
Merge branch 'feat/1813' into feat/1755
williballenthin e1186c5
features: add BinExport2 declarations
williballenthin 3acdd28
BinExport2: initial skeleton of feature extraction
williballenthin ebdc5fc
main: remove references to wip BinExport2 code
williballenthin 4b039cd
changelog
williballenthin dfa6c28
main: rename first position argument "input_file"
williballenthin 8c2c486
main: linters
williballenthin a9e1fd9
main: move rule-related routines to capa.rules
williballenthin d8d7f28
main: extract routines to capa.loader module
williballenthin 256d478
add loader module
williballenthin f652b36
Merge branch 'master' of public.github.com:mandiant/capa into feat/1813
williballenthin 554df5c
loader: learn to load freeze format
williballenthin 2d190aa
freeze: use new cli arg handling
williballenthin 247209b
Update capa/loader.py
williballenthin 0a979a3
main: remove duplicate documentation
williballenthin 44faf00
Merge branch 'feat/1813' of public.github.com:mandiant/capa into feat…
williballenthin 4183f86
main: add doc about where some functions live
williballenthin d536b9a
scripts: migrate to new main wrapper helper functions
williballenthin 43bb9e7
scripts: port to main routines
williballenthin 786cbb8
main: better handle auto-detection of backend
williballenthin 0cb1f9e
scripts: migrate bulk-process to main wrappers
williballenthin b3b1943
scripts: migrate scripts to main wrappers
williballenthin 189ae24
main: rename *_from_args to *_from_cli
williballenthin 7d80c91
changelog
williballenthin 435a3ca
cache-ruleset: remove duplication
williballenthin 3b4d2f4
main: fix tag handling
williballenthin 2b86c6e
cache-ruleset: fix cli args
williballenthin e3c8018
cache-ruleset: fix special rule cli handling
williballenthin de45f20
scripts: fix type bytes
williballenthin 051b336
Merge remote-tracking branch 'origin/feat/1813' into feat/1755
williballenthin 9c61809
main: nicely format debug messages
williballenthin 10c2e87
helpers: ensure log messages aren't very long
williballenthin b97d6c5
flake8 config
williballenthin 7573c94
binexport2: formatting
williballenthin f01de85
loader: learn to load BinExport2 files
williballenthin 40bcb1a
main: debug log the format and backend
williballenthin edbebb7
Merge branch 'master' of public.github.com:mandiant/capa into feat/1755
williballenthin 453094b
elf: add more arch constants
williballenthin 6cb9175
binexport: parse global features
williballenthin dbdf33d
binexport: extract file features
williballenthin 9681c53
binexport2: begin to enumerate function/bb/insns
williballenthin d71d087
binexport: pass context to function/bb/insn extractors
williballenthin a7a6e53
binexport: linters
williballenthin 217a9a9
binexport: linters
williballenthin f236ff2
scripts: add script to inspect binexport2 file
williballenthin 2202dc7
inspect-binexport: fix xref symbols
williballenthin 265ffe1
inspect-binexport: factor out the index building
williballenthin 27f60f3
binexport: move index to binexport extractor module
williballenthin 5d510c1
binexport: implement ELF/aarch64 GOT/thunk analyzer
williballenthin 162a0e8
binexport: implement API features
williballenthin 3a943bf
binexport: record the full vertex for a thunk
williballenthin f318129
binexport: learn to extract numbers
williballenthin afbff1b
binexport: number: skipped mapped numbers
williballenthin eb72d41
binexport: fix basic block address indexing
williballenthin 24ebea8
binexport: rename function
williballenthin e9e93da
binexport: extract operand numbers
williballenthin a405d4c
binexport: learn to extract calls from characteristics
williballenthin 874fa89
binexport: learn to extract mnemonics
williballenthin 159a796
pre-commit: skip protobuf file
williballenthin a1ad2d0
binexport: better search for sample file
williballenthin 673048f
loader: add file extractors for BinExport2
williballenthin 0f5d47c
binexport: remove extra parameter
williballenthin ffce03b
new black config
williballenthin 249398a
binexport: index string xrefs
williballenthin 5e9b308
binexport: learn to extract bytes and strings
williballenthin 4645363
binexport: cache parsed PE/ELF
williballenthin 373e944
Merge branch 'master' into feat/1755
williballenthin d2c744a
binexport: handle Ghidra SYMBOL numbers
williballenthin 1835587
binexport2: handle binexport#78 (Ghidra only uses SYMBOL expresssions)
williballenthin af4882d
Merge branch 'master' into feat/1755
williballenthin 933c9b1
main: write error output to stderr, not stdout
williballenthin f067f77
scripts: add example detect-binexport2-capabilities.py
williballenthin fead3a6
detect-binexport2-capabilities: more documentation/examples
williballenthin cc135df
Merge branch 'master' into feat/1755
williballenthin 8387be5
elffile: recognize more architectures
williballenthin 457df8a
binexport: handle read_memory errors
williballenthin 03c5130
binexport: index flow graphs by address
williballenthin 8283e36
binexport: cleanup logging
williballenthin 5ea8826
binexport: learn to extract function names
williballenthin 6a54e06
binexport: learn to extract all function features
williballenthin 4b45156
binexport: learn to extract bb tight loops
williballenthin fdf5305
elf: don't require vivisect just for type annotations
williballenthin a95e46c
main: remove unused imports
williballenthin 1813091
rules: don't eagerly import ruamel until needed
williballenthin 365b712
loader: avoid eager imports of some backend-related code
williballenthin a20fef5
changelog
williballenthin 44b3d85
fmt
williballenthin d5c4f78
Merge branch 'lazy-imports' into feat/1755
williballenthin 5c417cc
binexport: better render optional fields
williballenthin 6e497ed
merge upstream
mike-hunhoff 0d9d4c6
fix merge conflicts
mike-hunhoff 9c66b3a
fix formatting
mike-hunhoff 59775b2
remove Ghidra data reference madness
mike-hunhoff d39358e
handle PermissionError when searching sample file for BinExport2 file
mr-tz c1243cd
handle PermissionError when searching sample file for BinExport2 file
mr-tz b1d9554
add Android as valid OS
mr-tz cd62b87
Merge branch 'feat/1755' of github.com:mandiant/capa into feat/1755
mr-tz 14ff189
inspect-binexport: strip strings
williballenthin 10291e7
inspect-binexport: render operands
williballenthin 2783b10
fix lints
williballenthin a7447ed
ruff: update config layout
williballenthin 82dd3d7
inspect-binexport: better align comments/xrefs
williballenthin 8ed84f1
Merge branch 'master' into feat/1755
mr-tz 221eaa2
use explicit search paths to get sample for BinExport file
mr-tz 85f72ec
add initial BinExport tests
mr-tz 172b66d
add/update BinExport tests and minor fixes
mr-tz b07b498
inspect-binexport: add perf tracking
williballenthin 14116f7
inspect-binexport: cache rendered operands
williballenthin 971ff49
Merge branch 'feat/1755' of github.com:mandiant/capa into feat/1755
williballenthin 26d4bad
lints
williballenthin 29c2cbd
do not extract number features for ret instructions
mr-tz 9543e46
Fix BinExport's "tight loop" feature extraction.
larchchen e59a64a
Merge pull request #2050 from larchchen/feat/1755
mr-tz 498ff72
inspect-binexport: better render data section
williballenthin 2321395
Merge branch 'feat/1755' of github.com:mandiant/capa into feat/1755
williballenthin f83da38
linters
williballenthin 992049d
main: accept --format=binexport2
williballenthin 0dea7a3
binexport: insn: add support for parsing bare immediate int operands
williballenthin 966e62d
binexport2: bb: fix tight loop detection
williballenthin dc8c7e8
binexport: api: generate variations of Win32 APIs
williballenthin f37dd70
lints
williballenthin b4558df
binexport: index: don't assume instruction index is 1:1 with address
williballenthin 9c99af9
be2: index instruction addresses
mike-hunhoff 1fea6ab
be2: temp remove bytes feature processing
mike-hunhoff a5f26e3
merge upstream
mike-hunhoff 8760481
binexport: read memory from an address space extracted from PE/ELF
williballenthin 37aca87
be2: resolve thunks to imported functions
mike-hunhoff 1fcb272
merge upstream
mike-hunhoff 89c9126
be2: check for be2 string reference before bytes/string extraction ov…
mike-hunhoff bf33db8
be2: remove unneeded check
mike-hunhoff 8050a2f
be2: do not process thunks
mike-hunhoff 45d39cf
merge upstream
mike-hunhoff 2924bc3
Merge branch 'master' into feat/1755
williballenthin 5390e1a
be2: insn: polish thunk handling a bit
williballenthin 9a9d5a2
be2: pre-compute thunk targets
mike-hunhoff 136cac9
merge upstream
mike-hunhoff 054f39b
Merge branch 'master' into feat/1755
williballenthin 48881a2
Merge branch 'master' into feat/1755
williballenthin 68e8506
Merge branch 'master' into feat/1755
mr-tz 8a3b267
parse negative numbers
mr-tz 0ad7aea
update tests to use Ghidra-generated BinExport file
mr-tz b364485
remove unused import
mr-tz 674a89b
black reformat
mr-tz 2b0cc2c
run tests always (for now)
mr-tz 51578ca
binexport: tests: fix test case
mike-hunhoff a80bcc7
binexport: extractor: fix insn lint
mike-hunhoff 510aed2
binexport: addressspace: use base address recovered from binexport file
mike-hunhoff 9066a21
Add nzxor charecteristic in BinExport extractor.
larchchen bba29f4
add tests, fix stack cookie detection
mr-tz 58a8118
test BinExport feature PRs
mr-tz b92eba7
reformat and fix
mr-tz 3ed0075
Merge pull request #2073 from larchchen/feat/1755
mr-tz 6449520
Merge branch 'master' into feat/1755
mr-tz acbbca2
complete TODO descriptions
mr-tz 70891e4
wip tests
mr-tz b578c4d
merge upstream
mike-hunhoff 1d25c45
merge upstream
mike-hunhoff cbe83dd
binexport: add typing where applicable (#2106)
mike-hunhoff bb4e892
binexport2: revert import names from BinExport2 proto
williballenthin 8deb280
fix stack offset numbers and disable offset tests
mr-tz a2dc855
xfail OperandOffset
mr-tz 36cb7d9
generate symbol variants
mr-tz f98465a
wip: read negative numbers
mr-tz fe2e80f
update tight loop tests
mr-tz 798894a
binexport: fix function loop feature detection
mike-hunhoff 9e94987
binexport: update binexport function loop tests
mike-hunhoff 45b7b59
binexport: fix lints and imports
mike-hunhoff e7e786c
binexport: add back assert statement to thunk calculation
mike-hunhoff 427aad4
binexport: update tests to use Ghidra binexport file
mike-hunhoff 78665fc
merge upstream
mike-hunhoff 6efb46e
binexport: add additional debug info to thunk calculation assert
mike-hunhoff 207a48e
Merge branch 'master' into feat/1755
williballenthin 869b2f6
binexport: update unit tests to focus on Ghidra
mike-hunhoff 76a9f06
Merge branch 'feat/1755' of github.com:mandiant/capa into feat/1755
mike-hunhoff d14ce78
Merge branch 'master' into feat/1755
mr-tz 8527a3e
merge upstream
mike-hunhoff b70225d
merge upstream
mike-hunhoff 2e555d6
binexport: merge upstream
mike-hunhoff efde143
Merge branch 'master' into feat/1755
williballenthin a46257b
binexport: merge upstream
mike-hunhoff f8b0f50
binexport: fix lints
mike-hunhoff 227fdeb
binexport: remove Ghidra symbol madness and fix x86/amd64 stack offse…
mike-hunhoff 446a500
binexport: use masking for Number features
mike-hunhoff 5836b36
binexport: ignore call/jmp immediates for intel architecture
mike-hunhoff dfda0de
binexport: check if immediate is a mapped address
mike-hunhoff 7260c29
binexport: emit offset features for immediates likely structure offsets
mike-hunhoff fc3be31
binexport: add twos complement wrapper insn.py
mike-hunhoff 21d2b99
binexport: add support for x86 offset features
mike-hunhoff 210f127
binexport: code refactor
mike-hunhoff 877134e
binexport: init refactor for multi-arch instruction feature parsing
mike-hunhoff eb64254
binexport: merge upstream
mike-hunhoff be5f49a
binexport: intel: emit indirect call characteristic
mike-hunhoff 08c3429
binexport: use helper method for instruction mnemonic
mike-hunhoff a388b71
binexport: arm: emit offset features from stp instruction
mike-hunhoff d74c1da
binexport: arm: emit indirect call characteristic
mike-hunhoff fe48a75
binexport: arm: improve offset feature extraction
mike-hunhoff c22f773
binexport: add workaroud for Ghidra bug that results in empty operand…
mike-hunhoff 1660f2a
binexport: merge upstream
mike-hunhoff a9b7713
binexport: skip x86 stack string tests
mike-hunhoff 5488d83
binexport: merge upstream
mike-hunhoff 65e320e
binexport: update mimikatz.exe_ feature count tests for Ghidra
mike-hunhoff 1f10519
core: loader: update binja import
mike-hunhoff 5d89c29
core: loader: update binja imports
mike-hunhoff a10efe0
binexport: arm: ignore number features for add instruction manipulati…
mike-hunhoff 1fa7f02
binexport: update unit tests
mike-hunhoff 5624d9f
binexport: arm: ignore number features for sub instruction manipulati…
mike-hunhoff c9d58a3
binexport: arm: emit offset features for add instructions
mike-hunhoff f4f39d2
binexport: merge upsream
mike-hunhoff 981e93a
binexport: remove TODO from tests workflow
mike-hunhoff afa0215
binexport: update CHANGELOG
mike-hunhoff bfbd4ad
binexport: remove outdated TODOs
mike-hunhoff 7c8f7c9
binexport: merge upstream
mike-hunhoff 5ea55d1
binexport: re-enable support for data references in inspect-binexport…
mike-hunhoff 80cbe2a
binexport: skip data references to code
mike-hunhoff 1de3617
binexport: merge upstream
mike-hunhoff 7123f1f
binexport: remove outdated TODOs
mike-hunhoff eaa8945
Merge branch 'master' into feat/1755
williballenthin 1a3e63f
Update scripts/inspect-binexport2.py
williballenthin 210ba48
Update CHANGELOG.md
williballenthin a4f849c
Update capa/helpers.py
williballenthin f0fc44e
Update capa/features/extractors/common.py
williballenthin 5b3962f
Update capa/features/extractors/binexport2/extractor.py
williballenthin 7b7a680
Update capa/features/extractors/binexport2/arch/arm/insn.py
mike-hunhoff 577577b
initial add
mr-tz 5fd16c8
test binexport scripts
mr-tz b1211d9
add tests using small ARM ELF
mr-tz c662176
add method to get instruction by address
mr-tz bf38f22
index instructions by address
mr-tz 3c97edc
adjust and extend tests
mr-tz 7142bf7
handle operator with no children bug
mr-tz 23f8541
Merge branch 'master' into feat/1755
mr-tz 8e29295
Merge branch 'feat/1755' into tests/add-binexport
mr-tz fb6c628
Merge branch 'master' into feat/1755
mr-tz 5756ecf
Merge branch 'feat/1755' into tests/add-binexport
mr-tz 2c8b3ff
Merge pull request #2340 from mandiant/tests/add-binexport
mr-tz 9eaaa13
Merge branch 'master' into feat/1755
mr-tz e4d1b04
binexport: use instruction address index
williballenthin 0a5cc8e
inspect binexport: handle lsl with no children
williballenthin c59ed87
binexport: consolidate expression tree logic into helpers
williballenthin b5ec35d
binexport: index instruction indices by address
williballenthin 38dab5c
binexport: introduce instruction pattern matching
williballenthin fabe67d
Merge branch 'master' into feat/1755
williballenthin 90d500c
binexport: helpers: fix missing comment words
williballenthin b675808
Merge branch 'master' into feat/1755
williballenthin 38dad41
binexport: update tests to reflect updated test files
mike-hunhoff 765f434
Merge branch 'feat/1755' of github.com:mandiant/capa into feat/1755
mike-hunhoff b21d1c0
remove testing of feature branch
mr-tz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.