Test failure with new Intel compilers #2700

cross85 · 2023-05-23T17:06:33Z

I'm building the 4.9.2 version with new intel compiler icx. Specifically in nczarr_test in the run_ncgen4.sh test
But if I use the icc compiler, it works ok. Is it any known bug or still not completely compatible with the new compilers?

I did a very simple build:
export CC=icc
./configure --prefix=$path_netcdf/netcdf --disable-shared --with-pic --enable-largefile --disable-filter-testing --disable-dap --disable-netcdf4

WardF · 2023-05-23T22:05:30Z

We do not typically test against the Intel compilers; I'll take a look at icx and see if I can determine what is going on.

HathewayWill · 2023-06-16T00:59:21Z

`############################## Install NETCDF-C Library ############################

cd $WRF_FOLDER/Downloads
tar -xzvf v4.9.2.tar.gz
cd netcdf-c-4.9.2/

these variables need to be set for the NetCDF-C install to work

export CPPFLAGS=-I$DIR/grib2/include
export LDFLAGS=-L$DIR/grib2/lib
export LIBS="-lhdf5_hl -lhdf5 -lz -lcurl -lgfortran -lgcc -lm -ldl -lpnetcdf"

autoreconf -i

CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS ./configure --prefix=$DIR/NETCDF --disable-dap --enable-netcdf-4 --enable-netcdf4 --enable-shared --enable-pnetcdf --enable-cdf5 --enable-parallel-tests | tee netcdf.configure.log

make -j $CPU_HALF_EVEN 2>&1 | tee netcdf.make.log
make -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log | tee netcdf.makeinstall.log

other libraries below need these variables to be set

export PATH=$DIR/NETCDF/bin:$PATH
export NETCDF=$DIR/NETCDF

echo " "`

This works for GNU but not intel

@WardF

HathewayWill · 2023-06-16T01:01:13Z

`############################## Install NETCDF-C Library ############################

cd $WRF_FOLDER/Downloads tar -xzvf v4.9.2.tar.gz cd netcdf-c-4.9.2/

these variables need to be set for the NetCDF-C install to work

export CPPFLAGS=-I$DIR/grib2/include export LDFLAGS=-L$DIR/grib2/lib export LIBS="-lhdf5_hl -lhdf5 -lz -lcurl -lgfortran -lgcc -lm -ldl -lpnetcdf"

autoreconf -i

CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS ./configure --prefix=$DIR/NETCDF --disable-dap --enable-netcdf-4 --enable-netcdf4 --enable-shared --enable-pnetcdf --enable-cdf5 --enable-parallel-tests | tee netcdf.configure.log

make -j $CPU_HALF_EVEN 2>&1 | tee netcdf.make.log make -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log | tee netcdf.makeinstall.log

other libraries below need these variables to be set

export PATH=$DIR/NETCDF/bin:$PATH export NETCDF=$DIR/NETCDF

echo " "`

This works for GNU but not intel

@WardF

config.log
netcdf.configure.log

HathewayWill · 2023-07-02T02:40:15Z

@WardF @cross85 @edwardhartnett

So I found a solution for NETCDF-C(4.9.2) annd NETCDF-Fortran (4.6.0)

  cd $WRF_FOLDER/Downloads
  tar -xzvf v4.9.2.tar.gz
  cd netcdf-c-4.9.2/
  autoreconf -i -f 2>&1 | tee autoreconf.log

  # these variables need to be set for the NetCDF-C install to work
  export CPPFLAGS=-I$DIR/grib2/include
  export LDFLAGS=-L$DIR/grib2/lib
  export LIBS="-lhdf5_hl -lhdf5 -lz -lcurl -lgcc -lm -ldl -lpnetcdf"

  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/NETCDF --disable-dap --enable-netcdf-4 --enable-netcdf4 --enable-shared --enable-pnetcdf --enable-cdf5 --enable-parallel-tests 2>&1 | tee configure.log
  automake -a -f 2>&1 | tee automake.log
  make -j $CPU_HALF_EVEN 2>&1 | tee make.log
  make -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log

  # other libraries below need these variables to be set
  export PATH=$DIR/NETCDF/bin:$PATH
  export NETCDF=$DIR/NETCDF

  echo " "
  ############################## NetCDF-Fortran library ############################

  cd $WRF_FOLDER/Downloads
  tar -xvzf v4.6.0.tar.gz
  cd netcdf-fortran-4.6.0/
  autoreconf -i -f 2>&1 | tee autoreconf.log

  # these variables need to be set for the NetCDF-Fortran install to work
  export LD_LIBRARY_PATH=$DIR/NETCDF/lib:$LD_LIBRARY_PATH
  export CPPFLAGS="-I$DIR/NETCDF/include -I$DIR/grib2/include"
  export LDFLAGS="-L$DIR/NETCDF/lib -L$DIR/grib2/lib"
  export LIBS="-lnetcdf -lpnetcdf -lcurl -lhdf5_hl -lhdf5 -lz -lm -ldl"
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/NETCDF --enable-netcdf-4 --enable-netcdf4 --enable-parallel-tests --enable-hdf5 2>&1 | tee configure.log
  automake -a -f 2>&1 | tee automake.log
  make -j $CPU_HALF_EVEN 2>&1 | tee make.log
  make  -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log


  echo " "

  export CC=icx
  export CXX=icpx
  export FC=ifx
  export F77=ifx
  export F90=ifx
  export MPIFC='mpiifort -fc=ifx'
  export MPIF77='mpiifort -fc=ifx'
  export MPIF90='mpiifort -fc=ifx'
  export MPICC='mpiicc -cc=icx'
  export MPICXX='mpiicpc -cxx=icpx'
  export CFLAGS="-fPIC -fPIE -O3 -Wno-implicit-function-declaration -Wno-absolute-value -Wno-incompatible-function-pointer-types -Wno-unknown-warning-option -Wno-unused-command-line-argument "
  export FFLAGS="-m64"
  export FCFLAGS="-m64"

Relevant Log Files:
NETCDF-C
make.log
config.log
automake.log
configure.log
autoreconf.log
make.install.log

NETCDF-Fortran:
make.log
config.log
automake.log
configure.log
autoreconf.log
make.install.log

Hope This helps.

MicroTed · 2024-03-05T19:14:26Z

Building with icx seems to work, but 'make check' runs into failures with infinities and NaNs. From tst_netcdf4.log:

*** Testing NaN
*** creating NaN test file tst_nans.nc...ok.
*** Tests successful!
6,7c6,7
<               fvar:_FillValue = nanf ;
<               fvar:att = -inff, nanf, inff ;
---
>               fvar:_FillValue = NaNf ;
>               fvar:att = -Infinityf, NaNf, Infinityf ;
9,10c9,10
<               dvar:_FillValue = nan ;
<               dvar:att = -inf, nan, inf ;
---
>               dvar:_FillValue = NaN ;
>               dvar:att = -Infinity, NaN, Infinity ;
13c13
<  fvar = -inf, _, _ ;
---
>  fvar = -Infinityf, _, Infinityf ;
15c15
<  dvar = -inf, _, _ ;
---
>  dvar = -Infinity, _, Infinity ;
FAIL tst_netcdf4.sh (exit status: 1)

Intel(R) oneAPI DPC++/C++ Compiler 2023.2.3 (2023.2.3.20231204)
netcdf-c at tag v4.9.2

WardF · 2024-03-05T19:21:21Z

So this error will take some thought of how to approach, as it appears to reflect a compiler-specific representation rather than an error in the code. Thank you for the updated report!

DennisHeimbigner · 2024-03-05T20:01:08Z

I was wrong. The decision to print "Infinity" or "NaN" is made by
the code in ncdump in the function pr_att_algs (and probably elsewhere in ncdump).
The relevant code looks roughly like this:

if(isnan(ff))
    printf("NaN");
else if(isinf(ff))
   printf("Infinity");

So my current hypothesis is that isnan() and isinf() are not working right
so the print in ncdump defaults to (roughly)

printf("%lf",ff);

Again, a simple program could test this.

It looks like printf() for icx is handling NaN and Infinity in a
different way than other compilers.
That is:

for icx: printf("%f",(float)nan) produces "nanf"
for e.g. gcc: printf("%f",(float)nan) produces "NaNf"

You easily create a program to test this. There is probably something somewhere
in the C language standard about this. I suspect that icx is wrong.

baneta · 2024-04-16T20:20:28Z

Building with icx seems to work, but 'make check' runs into failures with infinities and NaNs. From tst_netcdf4.log:

*** Testing NaN
*** creating NaN test file tst_nans.nc...ok.
*** Tests successful!
6,7c6,7
<               fvar:_FillValue = nanf ;
<               fvar:att = -inff, nanf, inff ;
---
>               fvar:_FillValue = NaNf ;
>               fvar:att = -Infinityf, NaNf, Infinityf ;
9,10c9,10
<               dvar:_FillValue = nan ;
<               dvar:att = -inf, nan, inf ;
---
>               dvar:_FillValue = NaN ;
>               dvar:att = -Infinity, NaN, Infinity ;
13c13
<  fvar = -inf, _, _ ;
---
>  fvar = -Infinityf, _, Infinityf ;
15c15
<  dvar = -inf, _, _ ;
---
>  dvar = -Infinity, _, Infinity ;
FAIL tst_netcdf4.sh (exit status: 1)

Intel(R) oneAPI DPC++/C++ Compiler 2023.2.3 (2023.2.3.20231204) netcdf-c at tag v4.9.2

I've just tried -fhonor-infinities compiler flag and I've passed all the tests. I'm not 100% sure if it's a great solution. I'm using version 2024.1.0.
This flag has following description: "Specify that floating-point optimizations are not allowed that assume arguments and results are not +-inf."

DennisHeimbigner · 2024-04-16T21:13:22Z

Did you try the test program I suggested to see what isinf() and isnan()
are doing?

acgvar · 2024-04-17T04:35:43Z

Thank you for the efforts. I've encountered this issue in make check just now. We're porting into a new machine. Currently, it seems -fhonor-infinities as suggested by @baneta passes the tests. I wonder about the implications for this. I'm no expert, but looking forward to its resolution.

HathewayWill · 2024-04-17T14:33:59Z

Building with icx seems to work, but 'make check' runs into failures with infinities and NaNs. From tst_netcdf4.log:
*** Testing NaN
*** creating NaN test file tst_nans.nc...ok.
*** Tests successful!
6,7c6,7
<               fvar:_FillValue = nanf ;
<               fvar:att = -inff, nanf, inff ;
---
>               fvar:_FillValue = NaNf ;
>               fvar:att = -Infinityf, NaNf, Infinityf ;
9,10c9,10
<               dvar:_FillValue = nan ;
<               dvar:att = -inf, nan, inf ;
---
>               dvar:_FillValue = NaN ;
>               dvar:att = -Infinity, NaN, Infinity ;
13c13
<  fvar = -inf, _, _ ;
---
>  fvar = -Infinityf, _, Infinityf ;
15c15
<  dvar = -inf, _, _ ;
---
>  dvar = -Infinity, _, Infinity ;
FAIL tst_netcdf4.sh (exit status: 1)
Intel(R) oneAPI DPC++/C++ Compiler 2023.2.3 (2023.2.3.20231204) netcdf-c at tag v4.9.2
I've just tried -fhonor-infinities compiler flag and I've passed all the tests. I'm not 100% sure if it's a great solution. I'm using version 2024.1.0. This flag has following description: "Specify that floating-point optimizations are not allowed that assume arguments and results are not +-inf."

seems more like a bandaid then a fix for the issue. Because it is supressing a warning right?

WardF · 2024-04-18T20:14:50Z

The failure being reported in the test appears to be a difference in infinity representation; thanks for reporting the flag -fhonor-inifinities; I'll need to dig more into this to see what's going on and the extent to which this is a potential issue vs. (my original assumption) a difference in compilers.

Maduvi · 2024-05-10T12:56:02Z

After reading this: abseil/abseil-cpp#1451 I was able to pass all "make check" tests by using the flag -fp-model=precise I hope this also helps.

WardF · 2024-05-10T16:16:59Z

Thank you all; @Maduvi I appreciate the reference, I will see about adding a compiler check and setting the flag appropriately in our CMakeLists.txt file!

WardF self-assigned this Mar 5, 2024

WardF added the area/compilers label Mar 5, 2024

WardF added this to the 4.9.3 milestone Mar 5, 2024

WardF mentioned this issue Jun 12, 2024

Add compiler flag to fix infinities issue with intel compilers. #2935

Merged

WardF closed this as completed in #2935 Jun 12, 2024

DennisHeimbigner mentioned this issue Aug 5, 2024

ncdump failure on intel oneapi compiler #2959

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failure with new Intel compilers #2700

Test failure with new Intel compilers #2700

cross85 commented May 23, 2023

WardF commented May 23, 2023

HathewayWill commented Jun 16, 2023

HathewayWill commented Jun 16, 2023

these variables need to be set for the NetCDF-C install to work

other libraries below need these variables to be set

HathewayWill commented Jul 2, 2023

MicroTed commented Mar 5, 2024

WardF commented Mar 5, 2024

DennisHeimbigner commented Mar 5, 2024 •

edited

Loading

baneta commented Apr 16, 2024

DennisHeimbigner commented Apr 16, 2024

acgvar commented Apr 17, 2024

HathewayWill commented Apr 17, 2024

WardF commented Apr 18, 2024

Maduvi commented May 10, 2024

WardF commented May 10, 2024

Test failure with new Intel compilers #2700

Test failure with new Intel compilers #2700

Comments

cross85 commented May 23, 2023

WardF commented May 23, 2023

HathewayWill commented Jun 16, 2023

these variables need to be set for the NetCDF-C install to work

other libraries below need these variables to be set

HathewayWill commented Jun 16, 2023

these variables need to be set for the NetCDF-C install to work

other libraries below need these variables to be set

HathewayWill commented Jul 2, 2023

MicroTed commented Mar 5, 2024

WardF commented Mar 5, 2024

DennisHeimbigner commented Mar 5, 2024 • edited Loading

baneta commented Apr 16, 2024

DennisHeimbigner commented Apr 16, 2024

acgvar commented Apr 17, 2024

HathewayWill commented Apr 17, 2024

WardF commented Apr 18, 2024

Maduvi commented May 10, 2024

WardF commented May 10, 2024

DennisHeimbigner commented Mar 5, 2024 •

edited

Loading