Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support up to LLVM 14 #1083

Merged
merged 4 commits into from
Apr 20, 2023
Merged

Support up to LLVM 14 #1083

merged 4 commits into from
Apr 20, 2023

Conversation

RyanGlScott
Copy link
Contributor

This adds support for LLVM versions up to 14 by:

In order to adapt to various test cases' output changing with Clang 13 and 14, I needed to do a minor rewrite of the way Clang version ranges are handled in crux-llvm's test suite. See the crux-llvm/test/README.md file for a description of the new conventions that are currently in use.

@RyanGlScott
crux-llvm-test: Overhaul treatment of Clang version ranges
221e1c1 
This is a minor rewrite of the logic used to check for Clang version ranges in
`crux-llvm` test cases. The new approach is similar to Case 3/3a from this
`tasty-sugar` document:
https://github.com/kquick/tasty-sugar/blob/1fc06bee124e02f49f6478bc1e1df13704cc4916/Ranges.org#case-3---explicit-and-a-weaker-match

In particular:

* We have adopted the convention that the test output for the most recent
  version of Clang is always contained in a bare `.good` file.
* There are no longer any `at-least-clang*` files, just `pre-clang*`. This
  greatly simplifies the number of cases to consider and the number of checks
  to implement.
* We now skip test configurations by having `SKIP_TEST` as the first line of the
  `.good` file. Again, this greatly simplifies the logic needed to skip test
  cases on certain configurations.
@RyanGlScott
Support up to LLVM 14
43a6767 
This adds support for LLVM versions up to 14 by:

* Bumping the `llvm-pretty` submodule to bring in the changes from
  GaloisInc/llvm-pretty#109.
* Bumping the `llvm-pretty-bc-parser` submodule to bring in the changes from
  GaloisInc/llvm-pretty-bc-parser#219.
* Updating the test output in `crux-llvm` and `uc-crux-llvm`'s test suites
  accordingly.
@RyanGlScott
Copy link
Contributor Author

Somewhat surprisingly, two test configurations fail with these crux-llvm test suite changes. Here are my diagnoses:

  • macOS 12: My changes to the crux-llvm test suite evidently made things more permissive, as we now attempt to run the full test suite on macOS, even though that we aren't correctly detecting the clang version on macOS. To be honest, I think I like this new behavior better, as we no longer silently discard all of the test cases. Still, we aren't quite equipped to run the crux-llvm test suite with macOS yet (see MacOS missing overrides for symbolic IO #885 and CI accidentally skips over crux-llvm tests on macOS #999), so I think I will explicitly disable the crux-llvm test suite on the macOS CI job for now.

  • Ubuntu 20.04: This is a strange one. This is a consequence of the fact that clang --version gives different output depending on whether you use Ubuntu 20.04 or 22.04. On Ubuntu 22.04, the clang --version output is nice and clean:

    Ubuntu LLVM version 12.0.1

    But on Ubuntu 20.04, it's messier:

    Ubuntu clang version 12.0.0-3ubuntu1~20.04.5

    The versions library has a very hard time parsing things containing tildes, as I've reported in Parse messy version numbers involving tildes fosskers/versions#62. A consequence of this is that the crux-llvm test suite does not identify this Clang as being version 12, causing the wrong .good files to be picked.

    I'm not quite sure of the best way to fix this, but perhaps it would suffice to just drop any tilde tags from version numbers like the one above. It's a bit ad hoc, but then again, the version tag itself is ad hoc.

@RyanGlScott
CI: Explicitly disable crux-llvm tests on macOS
e55c3f1 
The tests were previously skipped on macOS due to a fluke (see
#999), but with the changes made to
the crux-llvm test suite, this no longer occurs. Let's explicitly disable the
tests on macOS until #885 is fixed.
@RyanGlScott
crux-llvm-test: Remove troublesome characters from version numbers
cff93a0 
We have seen Clang version numbers like `12.0.0-3ubuntu1~20.04.5` in the wild,
which the `versions` library has a difficult time parsing. To avoid this, we
scrub out uses of the `~` character, which is the thing that trips up the
parser. A bit ad hoc, but oh well.
@RyanGlScott RyanGlScott merged commit b4422dd into master Apr 20, 2023
@RyanGlScott RyanGlScott deleted the llvm-14 branch April 20, 2023 18:13
RyanGlScott added a commit to GaloisInc/saw-script that referenced this pull request Apr 20, 2023
@RyanGlScott
Support up to LLVM 14
5997bb2 
This adds support for LLVM versions up to 14 by:

* Bumping the `llvm-pretty` submodule to bring in the changes from
  GaloisInc/llvm-pretty#109.
* Bumping the `llvm-pretty-bc-parser` submodule to bring in the changes from
  GaloisInc/llvm-pretty-bc-parser#219.
* Bumping the `crucible` submodule to bring in the changes from
  GaloisInc/crucible#1083.
* Updating the surrounding documentation accordingly.
@RyanGlScott RyanGlScott mentioned this pull request Apr 20, 2023
Merged
RyanGlScott added a commit to GaloisInc/llvm-pretty-bc-parser that referenced this pull request Apr 23, 2023
@RyanGlScott
disasm-test: Overhaul treatment of LLVM version ranges
e76b9c6 
This is a minor rewrite of the logic used to check for LLVM version ranges in
`disasm-llvm` test cases. The new approach is similar to Case 3/3a from this
`tasty-sugar` document:
https://github.com/kquick/tasty-sugar/blob/1fc06bee124e02f49f6478bc1e1df13704cc4916/Ranges.org#case-3---explicit-and-a-weak
atch

In particular:

* We have adopted the convention that the test output for the most recent
  version of LLVM is always contained in a bare `.ll` file.
* There are no longer any `at-least-llvm*` files, just `pre-llvm*`. This
  greatly simplifies the number of cases to consider and the number of checks
  to implement.
* We now skip test configurations by having `SKIP_TEST` as the first line of the
  `.ll` file. Again, this greatly simplifies the logic needed to skip test
  cases on certain configurations.

This is heavily inspired by a similar change made in GaloisInc/crucible#1083.
RyanGlScott added a commit to GaloisInc/llvm-pretty-bc-parser that referenced this pull request Apr 24, 2023
@RyanGlScott
disasm-test: Overhaul treatment of LLVM version ranges
bf4f573 
This is a minor rewrite of the logic used to check for LLVM version ranges in
`disasm-llvm` test cases. The new approach is similar to Case 3/3a from this
`tasty-sugar` document:
https://github.com/kquick/tasty-sugar/blob/1fc06bee124e02f49f6478bc1e1df13704cc4916/Ranges.org#case-3---explicit-and-a-weaker-match

In particular:

* We have adopted the convention that the test output for the most recent
  version of LLVM is always contained in a bare `.ll` file.
* There are no longer any `at-least-llvm*` files, just `pre-llvm*`. This
  greatly simplifies the number of cases to consider and the number of checks
  to implement.
* We now skip test configurations by having `SKIP_TEST` as the first line of the
  `.ll` file. Again, this greatly simplifies the logic needed to skip test
  cases on certain configurations.

This is heavily inspired by a similar change made in GaloisInc/crucible#1083.
@RyanGlScott RyanGlScott mentioned this pull request Apr 24, 2023
Merged
RyanGlScott added a commit to GaloisInc/llvm-pretty-bc-parser that referenced this pull request May 3, 2023
@RyanGlScott
disasm-test: Overhaul treatment of LLVM version ranges
9db8694 
This is a minor rewrite of the logic used to check for LLVM version ranges in
`disasm-llvm` test cases. The new approach is similar to Case 3/3a from this
`tasty-sugar` document:
https://github.com/kquick/tasty-sugar/blob/1fc06bee124e02f49f6478bc1e1df13704cc4916/Ranges.org#case-3---explicit-and-a-weaker-match

In particular:

* We have adopted the convention that the test output for the most recent
  version of LLVM is always contained in a bare `.ll` file.
* There are no longer any `at-least-llvm*` files, just `pre-llvm*`. This
  greatly simplifies the number of cases to consider and the number of checks
  to implement.
* We now skip test configurations by having `SKIP_TEST` as the first line of the
  `.ll` file. Again, this greatly simplifies the logic needed to skip test
  cases on certain configurations.

This is heavily inspired by a similar change made in GaloisInc/crucible#1083.
RyanGlScott added a commit to GaloisInc/saw-script that referenced this pull request May 26, 2023
@RyanGlScott
Support up to LLVM 14
65e898e 
This adds support for LLVM versions up to 14 by:

* Bumping the `llvm-pretty` submodule to bring in the changes from
  GaloisInc/llvm-pretty#109.
* Bumping the `llvm-pretty-bc-parser` submodule to bring in the changes from
  GaloisInc/llvm-pretty-bc-parser#219.
* Bumping the `crucible` submodule to bring in the changes from
  GaloisInc/crucible#1083.
* Updating the surrounding documentation accordingly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@langston-barrett langston-barrett langston-barrett approved these changes

@kquick kquick kquick approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@RyanGlScott @kquick @langston-barrett