Dev: SSL Behaviour and Testing

Testing SSL Certs with MinIO

Set up some SSL certs:

mkdir -p ~/.minio/certs
cd /home/aseaton/.minio/certs
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./private.key -out ./public.crt

The cert will be self-signed which is quite useful for testing a bad cert.

You can also create an expired cert:

sudo faketime '2010-01-01 00:00:00' openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./private.key -out ./public.crt

Spin up a Minio server on Linux (ArcticDB does not work with MinIO hosted on Windows).

sudo docker run \
   -p 9000:9000 \
   -p 9001:9001 \
   --name minio \
   -v /home/aseaton/minio/data:/data \
   -v /home/aseaton/.minio/:/root/.minio/ \
   -e "MINIO_ROOT_USER=ROOTNAME" \
   -e "MINIO_ROOT_PASSWORD=CHANGEME123" \
   quay.io/minio/minio server /data --console-address ":9001"

Navigate to the Minio console in your browser (localhost:9091) and set up access keys.

Connection URIs to minio look like the below, but of course using your own access keys:

from arcticdb import Arctic
uri = 's3s://localhost:9000:tst?access=6pWBXggusk9MPFTpvj5G&secret=nUZ8JebcSUJGLS5XQz4zG5vk2ISIHHissIC1Xrmv'
ac = Arctic(uri)
ac.list_libraries()  # this call will fail with a bad SSL cert

VerifySSL Setting

Configure ArcticDB verify SSL setting with:

$env:ARCTICDB_S3Storage_VerifySSL_int=1  # verify
$env:ARCTICDB_S3Storage_VerifySSL_int=0  # noverify

$env:ARCTICDB_AWS_LogLevel_int=6

Note that noverify still checks that the cert is unexpired. This is a constraint of the AWS S3 C++ SDK.

Verification off just sets,

SECURITY_FLAG_IGNORE_CERT_CN_INVALID SECURITY_FLAG_IGNORE_UNKNOWN_CA

in the WinHttp options API, but eg SECURITY_FLAG_IGNORE_CERT_DATE_INVALID cannot be set via AWS SDK. So there is a limit to what verification you can disable.

In cUrl (used on Linux builds) the flag is more complete, and behaves like this:

        if (m_verifySSL)
        {
            curl_easy_setopt(connectionHandle, CURLOPT_SSL_VERIFYPEER, 1L);
            curl_easy_setopt(connectionHandle, CURLOPT_SSL_VERIFYHOST, 2L);

#if defined(ENFORCE_TLS_V1_3) && LIBCURL_VERSION_NUM >= 0x073400 // 7.52.0
            curl_easy_setopt(connectionHandle, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_3);
#elif defined(ENFORCE_TLS_V1_2) && LIBCURL_VERSION_NUM >= 0x072200 // 7.34.0
            curl_easy_setopt(connectionHandle, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2);
#else
            curl_easy_setopt(connectionHandle, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1);
#endif
        }
        else
        {
            curl_easy_setopt(connectionHandle, CURLOPT_SSL_VERIFYPEER, 0L);
            curl_easy_setopt(connectionHandle, CURLOPT_SSL_VERIFYHOST, 0L);
        }

Testing a storage's cert

In order to get the certificate that an endpoint offers you can do the following. In a Linux shell run,

openssl s_client -showcerts -connect <host>:<https port:443> </dev/null | sed -n -e '/-.BEGIN/,/-.END/ p' > certifs.pem

Then you can verify certifs.pem by copying it over to whatever host you are using ArcticDB on and using its tool, for example openssl verify on Linux.

You can see info about the certificate with,

openssl x509 -in public.crt -text

Note its serial number and issuer.

On Windows, you can use certutil:

certutil -dump public.pem  # parse the public cert
certutil -verify public.pem  # tell me whether my system trusts this cert