-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automate architecture synchronization with latest LLVM #1803
Conversation
MIPS Mapper Integration unify format style, closes rizinorg#2 Restored readme.md location ARM Arch & Synctools Usage Docs fix redundant header auto sync MIPS integration apply clang-format ignore .idea folder apply `black` reformat
SystemZ & XCore & Sparc & RISCV & PPC
Sparc PPC XCore SystemZ RISCV Instructions Printing & Mapping AArch64&Rizin disasm passing (with some target-specific generated table still needed - left unchecked for now) ARM&Rizin disasm passing Mips&Rizin disasm passing
Update generated files to match newest generator Invocation simplification, w.r.t. rizinorg#5 Turn on the build target
Resolve ARM Thumb decoding issue
Inst Printer Predicate fix architecture integrity check & ppc naming patch fixes mips naming & addressing issues
Cleanups & warning elimination
Finalizes problems on tests
Minor syncing README updates
It is also recommended to look into https://github.com/Phosphorus15/capstone-1/blob/auto-sync-integ/sync/SYNCING.md and https://github.com/Phosphorus15/capstone-1/blob/auto-sync-integ/sync/README.md for more information |
Very cool! Did you try this on the latest LLVM version, and was anything
broken?
|
Actually yes, the generated .inc files were based on the latest commit of llvm-project, and we tested it against cstest so it works very well on most test cases, except that LLVM drops PowerPC QPX support (so this table could not be gen-ed), and a couple of thumb2 instructions are missing. |
Can you provide some quick summary on what changed for x86, arm & arm64
with this PR? This brings latest instruction update to all of them?
|
did you base this PR on some development snapshot of llvm? since development snapshot of llvm may include some bugs or WIP, I think it is better to base on stable version, which is llvm 13.0 at the moment. i am just wondering if that is a lot of work to PR based that version now? |
@@ -891,8 +896,9 @@ size_t CAPSTONE_API cs_disasm(csh ud, const uint8_t *buffer, size_t size, uint64 | |||
mci.flat_insn->mnemonic[0] = '\0'; | |||
mci.flat_insn->op_str[0] = '\0'; | |||
#endif | |||
|
|||
debugln("trying into dissasm"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you remove the debug code?
please move sync/ directory into suite/ |
@Phosphorus15 could you please address these warnings too?
|
@@ -29,7 +29,7 @@ option(CAPSTONE_BUILD_CSTOOL "Build cstool" ON) | |||
option(CAPSTONE_USE_DEFAULT_ALLOC "Use default memory allocation functions" ON) | |||
option(CAPSTONE_ARCHITECTURE_DEFAULT "Whether architectures are enabled by default" ON) | |||
option(CAPSTONE_DEBUG "Whether to enable extra debug assertions" OFF) | |||
option(CAPSTONE_INSTALL "Generate install target" OFF) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was done like this on purpose to support FetchContent
workflows without doing custom hackery on the cache. Likely a good solution is to set it to CAPSTONE_ROOT_PROJECT
(and also set the other sensible defaults):
if(CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_SOURCE_DIR)
set(CAPSTONE_ROOT_PROJECT ON)
endif()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regardless, the change seems unrelated to the goal of this pr, so it should not be part of it.
This surely could be (by rebasing the tablegen backend onto llvm 13.0) done with no much of a problem, just that we'd lose the update of some new instructions on ARMv9 in the lastest dev branch of LLVM. |
x86 has not been included because it has a quite different tablegen backend implementation than other archs., also what is the |
It also should be rebased since there are now conflicts, @Phosphorus15 |
@XVilka emm..PR to 'next' branch is right, why need to rebase? |
@kabeor because there were recent commits in the |
@XVilka we will solve the conflict. 'next' branch is main branch for now. |
@Phosphorus15 Hello, is there any progress? or some thing I can help? |
@Phosphorus15 ping? |
I'm trying to introduce MIPS R6 support and just found this PR had done that :-) In case the original author stalled, is it possible to get this merged by maintainer edit? Or I can try to rebase and address these review comments, but how to credit the author properly? |
@FlyGoat I think you can do a new PR, it would be the best way to handle this. You can just keep original copyrights and add yours. You also could link to the original PR. Since the patched LLVM resides in our repository, feel free to send PRs here as well: https://github.com/rizinorg/llvm-capstone |
@FlyGoat Hello, we must keep the original code style, so we won't accept code formatting changes in this PR. In this PR, what we need to do are:
@XVilka For more convenient maintenance in the future, we really hope to move llvm-capstone repo to Capstone Organization. Maybe we can talk more about this. |
@kabeor yes, of course, that was the original intention once the PR is in the better shape to get merged. |
Cool! |
@kabeor I transferred the repository to you since cannot transfer to the capstone-engine directly. |
@XVilka ok, I'll transfer now, thx |
@XVilka hi, Would you like to maintain llvm-capstone continuing with us together? I can give you guys the access. |
For |
@FlyGoat yes |
@kabeor yes, please do. |
done :) |
I would take this PR and #1831 over and finish them up. If anyone already started on this as well in the last last 5 months please ping me here. |
What is Auto-Sync ?
Capstone Auto-Sync is an initiative to partly automate the synchronization of certain architectures to the latest.
Most of the Capstone's
.inc
files a generated from LLVM's TableGen backend and processed by python scrips insuite/synctools
into C-compatible files, which leads to the problem that with LLVM's update, it's not always (hardly, in fact) possible to use the synctools without patch in regard to LLVM's upstream change.This syncing tools, however, using a custom-made LLVM TableGen backend (here) to generate
.inc
files natively usable by Capstone. With certain adaptations in Capstone's structure, it is possible to consistently automate large parts of the work on keeping up with LLVM's latest Target (i.e..td
files) update, and optimally, there could be zero-overhead in the process of updating(see this patch on missing bcxf instructions)https://github.com/Phosphorus15/Capstone/blob/1bca4211bd7c132c57d0006a26de57d42a4bcdb9/sync/SYNCING.md?plain=1#L1-L17
What is done in this PR?
In this PR, we adapted following architectures with the Auto-Sync structure base on the
next
branch, and made sure they pass the test suite given by Capstone, and also by the analysis tools rizin (which also uses Capstone):What is yet to be done?
Capstone's instructions mapping info (like enums) exposed to various bindings (Python bindings, etc.) are not affected by Auto-Sync procedure, and so have to be manually edited if anything like new instructions/operands type was added by LLVM.