Installation issue: hdf-eos2

[danrosen25]

Steps to reproduce the issue

Download hdf-eos2-3.0-src.tar.gz from https://wiki.earthdata.nasa.gov/spaces/DAS/pages/202211788/Toolkit+Downloads#ToolkitDownloads-HDFEOSHDF-EOS

This can also be done via cloning the git repo containing this tarball
git clone -b HDFEOS2_3.0 https://git.earthdata.nasa.gov/scm/das/hdfeos.git
or using wget
wget "https://git.earthdata.nasa.gov/projects/DAS/repos/hdfeos/raw/hdf-eos2-3.0-src.tar.gz?at=refs%2Fheads%2FHDFEOS2_3.0 and then renaming the file

spack install hdf-eos2

 -   hdf-eos2@3.0+shared+static build_system=autotools patches:=5878c11 platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]      ^compiler-wrapper@1.0 build_system=generic platform=linux os=ubuntu25.04 target=m1 
[e]      ^gcc@14.2.0~binutils+bootstrap~graphite~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=ubuntu25.04 target=aarch64 
[+]      ^gcc-runtime@14.2.0 build_system=generic platform=linux os=ubuntu25.04 target=m1 
[e]      ^glibc@2.41 build_system=autotools platform=linux os=ubuntu25.04 target=aarch64 
[+]      ^gmake@4.4.1~guile build_system=generic platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]      ^gnuconfig@2024-07-27 build_system=generic platform=linux os=ubuntu25.04 target=m1 
[+]      ^hdf@4.2.15+external-xdr~fortran~java~netcdf+pic~shared~szip build_system=autotools patches:=49733dd,5434f29,ce75518,d54592d,f7d67e8 platform=linux os=ubuntu25.04 target=m1 %c,cxx,fortran=gcc@14.2.0
[+]          ^bison@3.8.2~color build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]              ^diffutils@3.12 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                  ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]              ^m4@1.4.20+sigsegv build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                  ^libsigsegv@2.14 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]          ^flex@2.6.3+lex~nls build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]              ^findutils@4.10.0 build_system=autotools patches:=440b954 platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                  ^gettext@0.23.1+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                      ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                      ^libxml2@2.13.5~http+pic~python+shared build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                      ^tar@1.35 build_system=autotools zip=pigz platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                          ^pigz@2.8 build_system=makefile platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                          ^zstd@1.5.7+programs build_system=makefile compression:=none libs:=shared,static platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                      ^xz@5.6.3~pic build_system=autotools libs:=shared,static platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]          ^libtirpc@1.3.7 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]              ^krb5@1.21.3+shared build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                  ^libedit@3.1-20240808 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                  ^openssl@3.6.0~docs+shared build_system=generic certs=mozilla platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                      ^ca-certificates-mozilla@2025-08-12 build_system=generic platform=linux os=ubuntu25.04 target=m1 
[+]                  ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                      ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                      ^gdbm@1.25 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                          ^readline@8.3 build_system=autotools patches:=21f0a03 platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]                  ^pkgconf@2.5.1 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]      ^libjpeg-turbo@3.0.4~ipo~jpeg8~partial_decoder+pic build_system=cmake build_type=Release generator=make libs:=shared,static platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]          ^cmake@3.31.9~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]              ^curl@8.15.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]                  ^nghttp2@1.67.1 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]              ^ncurses@6.5-20250705~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0
[+]          ^nasm@2.16.03 build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c=gcc@14.2.0
[+]      ^zlib-ng@2.2.4+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=ubuntu25.04 target=m1 %c,cxx=gcc@14.2.0

Error message

Error message

==> Error: ProcessError: Command exited with status 1:
    '/tmp/root/spack-stage/spack-stage-hdf-eos2-3.0-wzdzrw5xzzrah3hv6sl2n4je3xszrrmi/spack-src/configure' '--prefix=/spack/opt/spack/linux-m1/hdf-eos2-3.0-wzdzrw5xzzrah3hv6sl2n4je3xszrrmi' '--with-pic' '--enable-install_include' '--enable-shared' '--enable-static' '--with-hdf4=/spack/opt/spack/linux-m1/hdf-4.2.15-fmplvrhx45byqot264zeh7cr6jpi5yot' '--with-jpeg=/spack/opt/spack/linux-m1/libjpeg-turbo-3.0.4-raidl2r74jsa2o42vqg2n5goa7xewwto/lib'

1 error found in build log:
     132    checking hdf.h usability... yes
     133    checking hdf.h presence... yes
     134    checking for hdf.h... yes
     135    checking mfhdf.h usability... yes
     136    checking mfhdf.h presence... yes
     137    checking for mfhdf.h... yes
  >> 138    configure: error: HDF4 library 'mfhdf' not found

Information on your system

I'm building from within a Docker image container ubuntu:plucky on MacOS Sonoma 14.7 with Apple M2 Pro

• Spack: 1.2.0.dev0 (spack/spack@8a47f8d)
• Builtin repo: 38a07b7
• Python: 3.13.3
• Platform: linux-ubuntu25.04-m1

docker run --rm -it ubuntu:plucky
apt-get update && apt-get install -y file bzip2 ca-certificates g++ gcc gfortran git gzip lsb-release patch python3 tar unzip xz-utils zstd
git clone --depth=2 https://github.com/spack/spack.git
cd spack
. share/spack/setup-env.sh
git clone -b HDFEOS2_3.0 https://git.earthdata.nasa.gov/scm/das/hdfeos.git
mv hdfeos/hdf-eos2-3.0-src.tar.gz .
spack install hdf-eos2

Additional information

spack-build-env.txt
spack-build-out.txt

@climbfuji
Have you had success installing hdf-eos2@3.0 using spack with gcc@14.2.0? Is there a separate system issue that I'm missing?

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
• I have uploaded the build log and environment files
• I have searched the issues of this repo and believe this is not a duplicate

[climbfuji]

@danrosen25 This is from last year: JCSDA/spack-stack#1113 and spack/spack#44168

[danrosen25]

@danrosen25 This is from last year: JCSDA/spack-stack#1113 and spack/spack#44168

@climbfuji So it sounds like spack is not going to support installation of hdf-eos2 on any system? The error I'm seeing is configure: error: HDF4 library 'mfhdf' not found and it's technically on a Linux operating system with gcc@14, even though the chip is aarch64. I'm working with the LISF team to enhance portability of LIS coupling system. Maybe I'll try to resolve it myself and submit a PR for you to review.

Also, I didn't see any issue for hdf-eos2 in this repo, spack/spack-packages, and didn't think to check spack/spack.

[climbfuji]

This bug was reported long before the separation of the spack package repo from the spack repo occurred, that's why the issue is in the spack repository.

spack is a community effort and relies on users and developers who need a functionality to implement, maintain, and fix it if broken. If you need this package and apparently nobody else is using it, then please provide a fix for it. I am happy to review.

Somewhat related, I got this email a few weeks ago from @edwardhartnett:

I've just released a new version of the NetCDF Expansion Pack - in it I add a reader for NASA CDF data.
NASA CDF format is used for some NASA missions, and the CDF reader allows such files to be read as if they were netCDF files.
Get the release here: https://github.com/Intelligent-Data-Design-Inc/NEP/releases

Maybe this is a good opportunity to finally bury the outdated hdf-eos2 code and develop an expansion pack for NetCDF to read those files?

[danrosen25]

@climbfuji
Thanks, I'll bring it up with the LISF team and they will have to make the call if the expansion pack satisfies their needs. In the meantime I'll try to fix up the spack installation for hdf-eos2 but I'll have to put a time box on it. Thanks for your help!

[danrosen25]

Hi Dom @climbfuji,
I had lots of trouble linking hdf-eos2 to hdf4. For some reason --with-hdf4=DIR doesn't work. Right now it's set to self.spec["hdf"].prefix but I also tried the lib directory using self.spec["hdf"].libs.directories and it still can't find the mfhdf library. I have a work around to use h4cc based on the spack hdf installation (see below). Let me know your thoughts before I submit a pull request. Also gcc@14 now errors on implicit-int and implicit-function-declaration.

diff --git a/repos/spack_repo/builtin/packages/hdf_eos2/package.py b/repos/spack_repo/builtin/packages/hdf_eos2/package.py
index 041dad5d..da962bd7 100644
--- a/repos/spack_repo/builtin/packages/hdf_eos2/package.py
+++ b/repos/spack_repo/builtin/packages/hdf_eos2/package.py
@@ -125,7 +125,7 @@ class HdfEos2(AutotoolsPackage):
     def flag_handler(self, name, flags):
         if name == "cflags":
             flags.append(self.compiler.cc_pic_flag)
-            if self.spec.compiler.name in ["apple-clang", "oneapi"]:
+            if self.spec.compiler.name in ["apple-clang", "oneapi", "gcc"]:
                 flags.append("-Wno-error=implicit-function-declaration")
                 flags.append("-Wno-error=implicit-int")
 
@@ -140,6 +140,12 @@ class HdfEos2(AutotoolsPackage):
             extra_ldflags.append(tmp)
         # Set LDFLAGS
         env.set("LDFLAGS", " ".join(extra_ldflags))
+        # Use h4cc compiler wrapper and flags
+        if self.spec.satisfies("^hdf"):
+            tmp = self.spec["hdf"].prefix
+            env.set("CC", os.path.join(tmp.bin, "h4cc"))
+            if self.spec.compiler.name in ["apple-clang", "oneapi", "gcc"]:
+                env.set("CFLAGS", "-Wno-error=implicit-function-declaration -Wno-error=implicit-int")
 
     def configure_args(self):
         extra_args = []

[climbfuji]

You'll probably want to update the check for the compiler and version so that the implicit-int flags don't get applied for older versions of gcc that don't support it?

Using h4cc is subideal, but if it works so be it

[danrosen25]

@climbfuji That sounds good. I'm going to submit this PR. The LISF team utilizes the hdf-eos2 fortran interface so I also added it as a variant. And I cleaned up the --with-<lib>= arguments to point to the library directories.

Do you want me to also add myself as a maintainer?

diff --git a/repos/spack_repo/builtin/packages/hdf_eos2/package.py b/repos/spack_repo/builtin/packages/hdf_eos2/package.py
index 041dad5d..a2349038 100644
--- a/repos/spack_repo/builtin/packages/hdf_eos2/package.py
+++ b/repos/spack_repo/builtin/packages/hdf_eos2/package.py
@@ -66,17 +66,19 @@ class HdfEos2(AutotoolsPackage):
     variant(
         "static", default=True, description="Build static libraries (can be used with +shared)"
     )
+    variant("fortran", default=False, description="Enable Fortran support")
 
     conflicts("~static", when="~shared", msg="At least one of +static or +shared must be set")
 
     depends_on("c", type="build")
+    depends_on("fortran", type="build", when="+fortran")
 
     # Build dependencies
     depends_on("hdf")
     # Because hdf always depends on zlib and jpeg in spack, the tests below in configure_args
     # (if self.spec.satisfies("^jpeg"):) always returns true and hdf-eos2 wants zlib and jpeg, too.
-    depends_on("zlib-api")
-    depends_on("jpeg")
+    depends_on("zlib-api", when="^hdf")
+    depends_on("jpeg", when="^hdf")
     depends_on("szip", when="^hdf +szip")
 
     # Fix some problematic logic in stock configure script
@@ -125,7 +127,12 @@ class HdfEos2(AutotoolsPackage):
     def flag_handler(self, name, flags):
         if name == "cflags":
             flags.append(self.compiler.cc_pic_flag)
-            if self.spec.compiler.name in ["apple-clang", "oneapi"]:
+            if (
+                self.spec.satisfies("%clang@16:")
+                or self.spec.satisfies("%apple-clang@15:")
+                or self.spec.satisfies("%oneapi")
+                or self.spec.satisfies("%gcc@14:")
+            ):
                 flags.append("-Wno-error=implicit-function-declaration")
                 flags.append("-Wno-error=implicit-int")
 
@@ -140,6 +147,17 @@ class HdfEos2(AutotoolsPackage):
             extra_ldflags.append(tmp)
         # Set LDFLAGS
         env.set("LDFLAGS", " ".join(extra_ldflags))
+        # Use h4cc compiler wrapper and flags
+        if self.spec.satisfies("^hdf"):
+            tmp = self.spec["hdf"].prefix
+            env.set("CC", os.path.join(tmp.bin, "h4cc"))
+            if (
+                self.spec.satisfies("%clang@16:")
+                or self.spec.satisfies("%apple-clang@15:")
+                or self.spec.satisfies("%oneapi")
+                or self.spec.satisfies("%gcc@14:")
+            ):
+                env.set("CFLAGS", "-Wno-error=implicit-function-declaration -Wno-error=implicit-int")
 
     def configure_args(self):
         extra_args = []
@@ -152,17 +170,27 @@ class HdfEos2(AutotoolsPackage):
         extra_args.extend(self.enable_or_disable("shared"))
         extra_args.extend(self.enable_or_disable("static"))
 
+        # Set fortran
+        extra_args.extend(self.enable_or_disable("fortran"))
+
         # Provide config args for dependencies
-        extra_args.append("--with-hdf4={0}".format(self.spec["hdf"].prefix))
+        if self.spec.satisfies("^hdf"):
+            tmp = self.spec["hdf"].libs.directories
+            if tmp:
+                extra_args.append("--with-hdf4={0}".format(tmp[0]))
         if self.spec.satisfies("^jpeg"):
             # Allow handling whatever provider of jpeg are using
             tmp = self.spec["jpeg"].libs.directories
             if tmp:
-                tmp = tmp[0]
-                extra_args.append("--with-jpeg={0}".format(tmp))
+                extra_args.append("--with-jpeg={0}".format(tmp[0]))
         if self.spec.satisfies("^szip"):
-            extra_args.append("--with-szlib={0}".format(self.spec["szip"].prefix))
-        if self.spec.satisfies("^zlib"):
-            extra_args.append("--with-zlib={0}".format(self.spec["zlib-api"].prefix))
+            tmp = self.spec["szip"].libs.directories
+            if tmp:
+                extra_args.append("--with-szlib={0}".format(tmp[0]))
+        if self.spec.satisfies("^zlib-api"):
+            # Allow handling whatever provider of zlib-api are using
+            tmp = self.spec["zlib-api"].libs.directories
+            if tmp:
+                extra_args.append("--with-zlib={0}".format(tmp[0]))
 
         return extra_args

[climbfuji]

@climbfuji That sounds good. I'm going to submit this PR. The LISF team utilizes the hdf-eos2 fortran interface so I also added it as a variant. And I cleaned up the --with-<lib>= arguments to point to the library directories.

Do you want me to also add myself as a maintainer?

That would be awesome, thanks!