-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inconsistency in w4-yices/What4 - incorrect SMTLib-2 giving rise to false counterexamples. #1210
Comments
An update: on some experimentation, I've managed to find this (possible minimal) example:
With demonstration:
The generated SMT-Lib files are as follows, using SBV and What4 in order:
Obviously this example is absurd (you would never do this) but I think it is the same core issue as the one above (with all of my context removed - and this did naturally come up in the context of verifying some crypto code). |
and
produce different SMT-Lib code using SBV, but identical SMT-Lib code using What4. Of course, the first is always true and the second is not. Note that if you remove the dependence on i from the second term, or replace r@0 with simply a 1-bit array, there's no issue - something appears to be breaking in that interaction. |
Is it possible this is related to this: #1191 |
This appears to be different than #1191, unfortunately. |
The following also demonstrates the problem, and doesn't require any symbolic variables.
The issue seems to arise only when |
Yup, looks like this line: Pretty sure that |
And, indeed, if I change |
The underlying bug is fixed via GaloisInc/what4#127 I'll leave this ticket open until we get a new What4 release and update Cryptol to use it. |
I have the following property:
When I set prover = yices, and ask Cryptol to :prove bitgit, I get a QED in a handful of seconds. Similarly, z3 and mathsat also work. However, when I set prover = w4-yices (or indeed w4-z3), I get a "Counterexample", and then a counterexample is provided. This counterexample is not actually a counterexample, however - as in, when I run bitget on the data it says bitget evaluates as False, it reads True.
Obviously this is a pretty serious issue - I assume the What4 backend is producing wrong SMTLIB-2 somehow. I wasn't sure if I should raise this here or in https://github.com/GaloisInc/what4 but I've put it here for now. I have to assume - having found this - that I cannot trust the output of w4- proofs until it is fixed - unless anyone can satiate my fears?
I've included the output so you can see exactly what I mean.
The text was updated successfully, but these errors were encountered: