Fall back to Z3 4.8.8 on AWSLC/BLST proofs

[RyanGlScott]

This updates the what4-solvers snapshot to one that includes both Z3 4.8.8 and 4.8.14. While we continue to use 4.8.14 for most CI purposes, we fall back to 4.8.8 specifically when running the AWSLC or BLST proofs, which have been known to nondeterministically time out with those versions. Hopefully, this should resolve our CI woes.

This avoids the main issue in #1772, although the underlying cause still has yet to be investigated.

[RyanGlScott]

Hm, I need to figure out where the what4-solvers binaries are actually located on the awslc job. Currently, that job fails with:

+ rm bin/saw
+ cp /saw-bin/saw bin/saw
+ cp /saw-bin/abc bin/abc
+ cp /saw-bin/z3-4.8.10 bin/z3
cp: cannot stat '/saw-bin/z3-4.8.10': No such file or directory
1
make: *** [Makefile:25: awslc] Error 1

(cp /saw-bin/z3-4.8.10 bin/z3 was an educated guess, but an incorrect one.)

[RyanGlScott]

Ah, this is because I hadn't changed .github/ci.sh to actually copy over z3-4.8.10 into the uploaded artifact. After doing that, the AWSLC proofs now use z3-4.8.10, and the result is:

[15:10:02.875] Checking proof obligations ECDSA_do_verify ...
Killed
137
make: *** [Makefile:25: awslc] Error 137
Error: Process completed with exit code 2.

Ugh.

[RyanGlScott]

I realized something while looking at the entrypoint script for AWSLC:

saw-script/s2nTests/scripts/awslc-entrypoint.sh

Lines 1 to 13 in f9b6194

	#!/usr/bin/env bash
	set -xe
	cd /saw-script/aws-lc-verification/SAW
	./scripts/install.sh
	rm bin/saw
	cp /saw-bin/saw bin/saw
	cp /saw-bin/abc bin/abc
	export PATH=/saw-script/aws-lc-verification/SAW/bin:$PATH
	export CRYPTOLPATH=/saw-script/aws-lc-verification/cryptol-specs
	./scripts/entrypoint_check.sh

Namely, this was never copying over the what4-solvers version of Z3 in the first place! It copies over the what4-solvers version of abc (which lives under /saw-bin) to /bin (which is what is put onto the PATH), but no other solver receives this treatment. The remaining solvers come from the install.sh script, which lives in the aws-lc-verification repo:

Z3_URL='https://github.com/Z3Prover/z3/releases/download/z3-4.8.8/z3-4.8.8-x64-ubuntu-16.04.zip'

# ...

# fetch Z3
if [ ! -f bin/z3 ]
then
    mkdir -p deps/z3
    wget $Z3_URL -O deps/z3.zip
    unzip deps/z3.zip -d deps/z3
    cp deps/z3/*/bin/z3 bin/z3
fi

This is a double whammy: not only was the AWSLC script installing a different version of Z3 than we originally expected (4.8.8), it was installing the Ubuntu 16.04 version! It is a minor miracle that this continued to work for as long as it did.

Unfortunately, this PR has demonstrated that using the Ubuntu 22.04 version of Z3 4.8.10 with the AWSLC proofs results in the OOM killer acting up. On the other hand, we have never actually tried the Ubuntu 22.04 version of Z3 4.8.14, so I will push a change that tries out 4.8.14 on the AWSLC proofs.

---

While I was in town, I noticed that the BLST entrypoint script is somewhat overwrought:

saw-script/s2nTests/scripts/blst-entrypoint.sh

Lines 6 to 10 in f9b6194

	cp /saw-bin/saw bin/saw
	wget --quiet -O solvers.zip "https://github.com/GaloisInc/what4-solvers/releases/download/snapshot-20220902/ubuntu-22.04-bin.zip"
	(cd bin && unzip -o ../solvers.zip)
	chmod +x bin/*

This re-downloads the what4-solvers binaries, but there is no point in doing so, as they are already available under /bin. I will push a separate commit to simplify this.

[RyanGlScott]

Sadly, the OOM killer still activates on the AWSLC proofs even with Z3 4.8.14. I suppose we could try using Z3 4.8.8 next, as that is the Z3 version that is used in aws-lc-verifications's install.sh script.

[RyanGlScott]

Well, the AWSLC/BLST proofs passed after downgrading to Z3 4.8.8. Let's see if this is consistent after restarting those jobs a couple of times...

[RyanGlScott]

With Z3 4.8.8, the AWSLC and BLST jobs have succeeded on three consecutive runs, which makes me cautiously optimistic. I've rebased everything, and assuming that this successful CI trend continues after this rebase, then I'm declaring victory.

[RyanGlScott]

Hot dog, I think it works! @chameco, does everything look good to you?

[chameco]

Thanks @RyanGlScott ! LGTM

[RyanGlScott]

@Mergifyio refresh

[mergify]

refresh

✅ Pull request refreshed