Error running ra-tls-secret-prov example

[sroy9gmu]

I am facing errors while following the steps outlined for running Secret Provisioning flows for EPID-based (IAS) attestation.

Currently, both client and server windows are running on the same machine (localhost). The relevant details of both windows are shown below

Server window:

COMMANDS

cd ra-tls-secret-prov
export RA_TLS_ALLOW_DEBUG_ENCLAVE_INSECURE=1
export RA_TLS_ALLOW_OUTDATED_TCB_INSECURE=1
export RA_TLS_ALLOW_HW_CONFIG_NEEDED=1
export RA_TLS_ALLOW_SW_HARDENING_NEEDED=1
make app epid RA_TYPE=epid RA_CLIENT_SPID=[spid for my machine] RA_CLIENT_LINKABLE=0
cd secret_prov_pf
RA_TLS_EPID_API_KEY=[api key for my machine] ./server_epid wrap_key &

OUTPUT

[1] 6068
--- Reading the master key for encrypted files from 'wrap_key' ---
--- Starting the Secret Provisioning server on port 4433 ---
client_connection: Secret Provisioning failed during mbedtls_ssl_handshake with error -80
client_connection: ra_tls_verify_callback_results:
attestation_scheme=0, err_loc=0,
client_connection: unknown attestation scheme!

---

Client window:

COMMANDS

cd ra-tls-secret-prov/secret_prov_pf
sudo gramine-sgx ./client

OUTPUT

Attributes:
mr_enclave: c0a12f2c976b3f54345d59c888101427e8236e7c11663653f3383494e495c435
mr_signer: 117286b0cb09baf6f1d8fe317b5597760f6f6e559022d8f83148a68d852da53e
isv_prod_id: 0
isv_svn: 0
attr.flags: 0000000000000006
attr.xfrm: 0000000000000007
mask.flags: ffffffffffffffff
mask.xfrm: fffffffffff9ff1b
misc_select: 00000000
misc_mask: ffffffff
modulus: fb75854b3a9c3fc170936080fcdce167...
exponent: 3
signature: 3caddf9fcf42da83a2edaa8048874e0c...
date: 2023-08-18
Gramine is starting. Parsing TOML manifest file, this may take some time...
Gramine detected the following insecure configurations:

• sgx.debug = true (this is a debug enclave)

Gramine will continue application execution, but this configuration must not be used in production!

secret_provision_start: Secret Provisioning failed during mbedtls_ssl_handshake with error -9984
secret_provision_constructor: Secret provisioning failed, terminating the whole process

---

Note that I am able to run ra-tls-mbedtls example correctly using these same two windows as shown below.

Server window:

COMMANDS

cd ra-tls-mbedtls
sudo gramine-sgx ./server &

OUTPUT

Gramine is starting. Parsing TOML manifest file, this may take some time...
Gramine detected the following insecure configurations:

• sgx.debug = true (this is a debug enclave)
• loader.insecure__use_cmdline_argv = true (forwarding command-line args from untrusted host to the app)
• sgx.allowed_files = [ ... ] (some files are passed through from untrusted host without verification)

Gramine will continue application execution, but this configuration must not be used in production!

. Seeding the random number generator... ok

. Creating the RA-TLS server cert and key (using "epid" as attestation type)... ok
. Bind on https://localhost:4433/ ... ok
. Setting up the SSL data.... ok
. Waiting for a remote connection ...

---

Client window:

COMMANDS

cd ra-tls-mbedtls
RA_TLS_ALLOW_DEBUG_ENCLAVE_INSECURE=1 RA_TLS_ALLOW_OUTDATED_TCB_INSECURE=1 RA_TLS_EPID_API_KEY=[api key for my machine] RA_TLS_MRENCLAVE=d9c900be63fb6e893e42e25047961cab7debe4f5cb9e93e0fc3b04bfff8b6bb6 RA_TLS_MRSIGNER=117286b0cb09baf6f1d8fe317b5597760f6f6e559022d8f83148a68d852da53e RA_TLS_ISV_PROD_ID=0 RA_TLS_ISV_SVN=0 ./client epid

OUTPUT

[ using default SGX-measurement verification callback (via RA_TLS_* environment variables) ]

. Seeding the random number generator... ok
. Connecting to tcp/localhost/4433... ok
. Setting up the SSL/TLS structure... ok
. Loading the CA root certificate ... ok
. Installing RA-TLS callback ... ok
. Performing the SSL/TLS handshake...IAS report: signature verified correctly
IAS report: allowing quote status GROUP_OUT_OF_DATE
[ advisory URL: https://security-center.intel.com ]
[ advisory IDs: ["INTEL-SA-00106", "INTEL-SA-00115", "INTEL-SA-00135", "INTEL-SA-00203", "INTEL-SA-00220", "INTEL-SA-00233", "INTEL-SA-00270", "INTEL-SA-00293", "INTEL-SA-00320", "INTEL-SA-00329", "INTEL-SA-00334", "INTEL-SA-00381", "INTEL-SA-00389", "INTEL-SA-00477", "INTEL-SA-00614", "INTEL-SA-00615", "INTEL-SA-00617"] ]
ok
. Verifying peer X.509 certificate... ok
Write to server: 18 bytes written

GET / HTTP/1.0

Read from server: 158 bytes read

HTTP/1.0 200 OK
Content-Type: text/html

mbed TLS Test Server
Successful connection using: TLS-ECDHE-ECDSA-WITH-CHACHA20-POLY1305-SHA256

---

Please provide your input on resolving the secret provisioning example errors.

[kailun-qin]

@sroy9gmu Thanks for your question and information!

-9984 (-0x2700) indicates that the certificate verification failed, e.g. CRL, CA or signature check failed (pls see: https://github.com/Mbed-TLS/mbedtls/blob/1fdc884ed831927600838a3656b6d74a0417d52b/include/mbedtls/x509.h#L83).

However, there are several reasons for certificate verification to fail (e.g., incorrect trusted root CA certificate, the server hostname is different from the certificate CN in SN\SAN...). Since both client and server windows are running on the same machine (localhost) and you are able to run ra-tls-mbedtls example correctly using these same two windows, I don't see anything suspicious obviously and can't tell w/o further debug info.

Would you pls first try to clean up the SSL data from CI-Examples/ra-tls-secret-prov by e.g., make distclean; then re-build and re-run the example? If the issue persists, I suggest to check the verification flags by mbedtls_ssl_get_verify_result() or enbale mbedTLS debug logs. Thanks!

[sroy9gmu]

I apologize for the late reply. I cloned the gramine repo again, rebuild and reinstalled the binaries, and did a distclean of the ra-tls-secret-prov build outputs. Still, the issue persists. Probably I need to investigate further using debug messages like you suggested. I will reply back to this thread at a later point of time once I get further debug logs.