test_byterange.sh fails with BADCERT_NOT_TRUSTED

[Alexander-Barth]

• the version of the software with which you are encountering an issue

NetCDF 4.9.0

• environmental information (i.e. Operating System, compiler info, java version, python version, etc.)

Linux, x86_64-linux-gnu-gcc (GCC) 4.8.5 (no cross-compilation this time)

• a description of the issue with the steps needed to reproduce it

Build steps:

./configure --prefix=/workspace/destdir --build=x86_64-linux-musl --host=x86_64-linux-gnu \
   --enable-shared --disable-static --disable-dap-remote-tests --enable-byterange

When activating byte range support the ./test_byterange.sh tests fails with

curlcode: (60)SSL peer certificate or SSH remote key was not OK : Cert verify failed: BADCERT_NOT_TRUSTED
/workspace/srcdir/netcdf-c-4.9.0/ncdump/.libs/lt-ncdump: https://thredds-test.unidata.ucar.edu/thredds/fileServer/pointData/cf_dsg/example/point.nc#mode=bytes&aws.profile=none: NetCDF: libcurl failure
test_http: -k flag mismatch: expected=classic have=

This is related to issue #2337 but not necessarily a duplicate since I have defined a .ncrc file with a HTTP.SSL.CAINFO in /workspace/srcdir/netcdf-c-4.9.0/nc_test/.ncrc and /root/.ncrc:

# cat /root/.ncrc
HTTP.SSL.CAINFO=/etc/ssl/certs/ca-certificates.crt

# ls -lh /etc/ssl/certs/ca-certificates.crt
-r--r--r--    1 root     root      198.5K May  3 11:15 /etc/ssl/certs/ca-certificates.crt

# echo $HOME
/root

But if I hard code CURLOPT_CAINFO in dhttp.c, the first test with the url (https://thredds-test.unidata.ucar.edu/thredds/fileServer/pointData/cf_dsg/example/point.nc#mod e=bytes&aws.profile=none) passes.

--- /home/abarth/src/netcdf-c/libdispatch/dhttp.c	2022-06-14 10:20:57.826005241 +0200
+++ dhttp.c	2022-06-17 12:17:05.236041079 +0200
@@ -509,6 +509,8 @@
         (void)CURLERR(curl_easy_setopt(state->curl, CURLOPT_READDATA, NULL));
     }
 
+    (void)CURLERR(curl_easy_setopt(state->curl, CURLOPT_CAINFO, "/etc/ssl/certs/ca-certificates.crt"));
+    
     /* Do method specific actions */
     switch(state->request.method) {
     case HTTPPUT:

Could it be that the byte range support of NetCDF does not use HTTP.SSL.CAINFO ?

Unfortunately, there are even with this change other failures later on in ./test_byterange.sh:

/workspace/srcdir/netcdf-c-4.9.0/ncdump/.libs/lt-ncdump: https://thredds-test.unidata.ucar.edu/thredds/fileServer/irma/metar/files/METAR_20170910_0000.nc#bytes&aws.profile=none: NetCDF: HDF error

I am wondering if the HDF5 layer needs to be made aware of HTTP.SSL.CAINFO too.

[DennisHeimbigner]

The byterange driver I use is home-made because HDF5 does not
directly support it (AFAIK, I should recheck). So I will modify
our local code to explicitly set CAINFO from .rc tables.
I will post the modified code here for you to test.

[DennisHeimbigner]

PR #2408 should fix this,
assuming you set HTTP.CAINFO

[DennisHeimbigner]

This PR may fix this problem: #2408

[WardF]

#2408 is merged, now.