Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ufs hera update #192

Merged
merged 48 commits into from
Dec 12, 2022
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
b1a9a7a
Add patch in bacio to fix incorrect version in cmake files for v2.4.1
AlexanderRichert-NOAA Nov 8, 2022
74c1481
Add link flags to enable static libraries for ESMF
AlexanderRichert-NOAA Nov 8, 2022
44ff16b
Add soft links for ESMF libs to suit UFS capitalization scheme (-lESMF)
AlexanderRichert-NOAA Nov 8, 2022
a4fe5f6
Add 'shared' variant to allow static-only build for libpng
AlexanderRichert-NOAA Nov 8, 2022
ebd880a
Link zlib when compiling static netcdf-c
AlexanderRichert-NOAA Nov 8, 2022
622043e
Add variants for various build options
AlexanderRichert-NOAA Nov 8, 2022
63085f2
Add optional workaround for failed pfio test executables in MAPL
AlexanderRichert-NOAA Nov 8, 2022
1c0e6c5
Fix variant for patch to disable pfio test executables
AlexanderRichert-NOAA Nov 8, 2022
30bcd93
Add pnetcdf variant for MAPL
AlexanderRichert-NOAA Nov 8, 2022
488993f
Restructure ufs-weather-model-env package and add variant for shared …
AlexanderRichert-NOAA Nov 8, 2022
0121116
Add static build variant for netcdf-fortran
AlexanderRichert-NOAA Nov 8, 2022
05a1373
Merge branch 'NOAA-EMC:jcsda_emc_spack_stack' into ufs_hera_update
AlexanderRichert-NOAA Nov 8, 2022
fcac30f
Correct config_args aggregation in netcdf-c package
AlexanderRichert-NOAA Nov 8, 2022
a17c8e6
Fix when for parallelio in ufs-weather-model-env
AlexanderRichert-NOAA Nov 8, 2022
b2f3d34
ESMF package: remove caplinks variant and use nc-config to get link f…
AlexanderRichert-NOAA Nov 22, 2022
a52b696
HDF5 package: Patch incorrect pkg-config flags for zlib
AlexanderRichert-NOAA Nov 22, 2022
039ef9e
NetCDF-C package: Use pkg-config to get HDF5-related flags for static…
AlexanderRichert-NOAA Nov 22, 2022
66a778e
NetCDF-fortran package: Use pkg-config to get netcdf deps for static …
AlexanderRichert-NOAA Nov 22, 2022
552a2ed
UFS env: Remove versions
AlexanderRichert-NOAA Nov 22, 2022
c4ecf63
MAPL: Use nc-config to get netcdf flags for pfio executables; Remove …
AlexanderRichert-NOAA Nov 22, 2022
dae4706
merge
AlexanderRichert-NOAA Nov 22, 2022
f2f484a
MAPL: revert test changes
AlexanderRichert-NOAA Nov 22, 2022
304e22f
MAPL: revert accidental change
AlexanderRichert-NOAA Nov 22, 2022
cf163c2
Merge branch 'jcsda_emc_spack_stack' into ufs_hera_update
AlexanderRichert-NOAA Nov 22, 2022
d0c7393
UFS env: remove extraneous "f" string prefixes
AlexanderRichert-NOAA Nov 22, 2022
b57048b
UFS env: re-hardcode fms and crtm versions
AlexanderRichert-NOAA Nov 22, 2022
389e667
UFS env: Modify to resemble previous structure more closely; use base…
AlexanderRichert-NOAA Nov 23, 2022
283d541
Merge branch 'NOAA-EMC:jcsda_emc_spack_stack' into ufs_hera_update
AlexanderRichert-NOAA Nov 29, 2022
23bb728
Set ESMF shared/static based on UFS WM env switch
alexrichert Nov 29, 2022
5993219
Install python in base-env regardless of whether rest of python packa…
alexrichert Nov 29, 2022
5bfe007
Enable static netcdf-c for nccmp
alexrichert Nov 29, 2022
9e456a6
Finish removing python option from base-env
alexrichert Nov 30, 2022
be07091
Update UFS WM env package.py to fix base-env shared option
AlexanderRichert-NOAA Nov 30, 2022
683a6aa
Add shared/static parallelio variants in base-env
alexrichert Dec 1, 2022
19dde13
Correct doxygen default setting (=off) for netcdf-c
alexrichert Dec 2, 2022
69094bc
Fix nccmp w.r.t. netcdf-c+mpi
alexrichert Dec 2, 2022
46ce0d1
Merge branch 'NOAA-EMC:jcsda_emc_spack_stack' into ufs_hera_update
AlexanderRichert-NOAA Dec 8, 2022
80f56f3
Revert ufs-weather-model-env changes
AlexanderRichert-NOAA Dec 8, 2022
027a1ff
Revert base-env changes
AlexanderRichert-NOAA Dec 8, 2022
f88aaba
revert netcdf-c package.py as test
alexrichert Dec 8, 2022
804b038
Restore modified netcdf-c package.py, but revert unnecessary changes
alexrichert Dec 8, 2022
50a0cb0
switch hdf5/netcdf-c/netcdf-fortran
alexrichert Dec 9, 2022
1114656
revert hdf5 changes only
alexrichert Dec 9, 2022
a5ee2e7
restore netcdf-c changes
alexrichert Dec 9, 2022
d041920
Allow HDF5 zlib pkgconfig fix to work for shared zlib
alexrichert Dec 10, 2022
685b68c
Only use MPI compilers when needed for netcdf-fortran
alexrichert Dec 10, 2022
5a70ec9
Fix syntax netcdf-fortran package.py
alexrichert Dec 10, 2022
78e6013
Fix filter_file pattern for hdf5 pkgconfig
alexrichert Dec 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion var/spack/repos/builtin/packages/bacio/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@


class Bacio(CMakePackage):
"""The bacio ibrary performs binary I/O for the NCEP models, processing
"""The bacio library performs binary I/O for the NCEP models, processing
unformatted byte-addressable data records, and transforming the little
endian files and big endian files."""

Expand Down Expand Up @@ -39,3 +39,6 @@ def cmake_args(self):
args = [self.define_from_variant("CMAKE_POSITION_INDEPENDENT_CODE", "pic")]

return args

def patch(self):
filter_file('2\.4\.0',str(self.spec.version),"VERSION",when="bacio@2.4.1")
climbfuji marked this conversation as resolved.
Show resolved Hide resolved
10 changes: 10 additions & 0 deletions var/spack/repos/builtin/packages/esmf/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

import os
import subprocess

from spack.package import *

Expand Down Expand Up @@ -88,6 +89,7 @@ class Esmf(MakefilePackage):
when="@8.3.0b09",
)
variant("debug", default=False, description="Make a debuggable version of the library")
variant("shared", default=True, description="Build shared library")

# Required dependencies
depends_on("zlib")
Expand Down Expand Up @@ -310,6 +312,11 @@ def edit(self, spec, prefix):
# NetCDF format.
os.environ["ESMF_NETCDF"] = "nc-config"
os.environ["ESMF_NFCONFIG"] = "nf-config"
if spec["netcdf-c"].satisfies("~shared"):
nc_pc_cmd = ["nc-config","--static","--libs"]
nc_flags = \
subprocess.check_output(nc_pc_cmd, encoding="utf8").strip()
os.environ["ESMF_NETCDF_LIBS"] = nc_flags

###################
# Parallel-NetCDF #
Expand Down Expand Up @@ -356,6 +363,9 @@ def edit(self, spec, prefix):
# ESMF_XERCES_INCLUDE
# ESMF_XERCES_LIBPATH

if "~shared" in spec:
os.environ["ESMF_SHARED_LIB_BUILD"] = "OFF"

@run_after("install")
def install_findesmf(self):
install_tree("cmake", self.prefix.cmake)
Expand Down
6 changes: 6 additions & 0 deletions var/spack/repos/builtin/packages/hdf5/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,12 @@ def fix_package_config(self):
backup=False,
)

# HDF5 build incorrectly assign -l to zlib file path
filter_file(
"-l(/[^\s]+/libz\.a)", r"\1 -lz",
*pc_files, when="~shared"
)

# Create non-versioned symlinks to the versioned pkg-config files:
with working_dir(self.prefix.lib.pkgconfig):
for f in pc_files:
Expand Down
3 changes: 3 additions & 0 deletions var/spack/repos/builtin/packages/libpng/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ class Libpng(AutotoolsPackage):

depends_on("zlib@1.0.4:") # 1.2.5 or later recommended

variant("shared", default=True)

def configure_args(self):
args = [
# not honored, see
Expand All @@ -34,6 +36,7 @@ def configure_args(self):
f"CPPFLAGS={self.spec['zlib'].headers.include_flags}",
f"LDFLAGS={self.spec['zlib'].libs.search_flags}",
]
if "~shared" in self.spec: args += ["--enable-static","--disable-shared"]
return args

def check(self):
Expand Down
16 changes: 16 additions & 0 deletions var/spack/repos/builtin/packages/nccmp/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
from spack.package import *
import subprocess


class Nccmp(CMakePackage):
Expand All @@ -17,6 +18,7 @@ class Nccmp(CMakePackage):

depends_on("cmake@3.12:", type="build")
depends_on("netcdf-c", type=("build", "run"))
depends_on("mpi", when="^netcdf-c+mpi~shared")

def cmake_args(self):
args = []
Expand All @@ -28,4 +30,18 @@ def cmake_args(self):
if cflags:
args.append(self.define("CMAKE_C_FLAGS", " ".join(cflags)))

nc = self.spec["netcdf-c"]
if "~shared" in nc:
nc_pc_cmd = ["nc-config","--static","--libs"]
nc_flags = \
subprocess.check_output(nc_pc_cmd, encoding="utf8").strip()
args.append(self.define("CMAKE_EXE_LINKER_FLAGS", nc_flags))
if "+mpi" in nc:
args.append(
self.define(
"CMAKE_C_COMPILER",
self.spec["mpi"].mpicc
climbfuji marked this conversation as resolved.
Show resolved Hide resolved
)
)

return args
22 changes: 22 additions & 0 deletions var/spack/repos/builtin/packages/netcdf-fortran/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import glob
import os
from shutil import Error, copyfile
import subprocess

from spack.package import *

Expand Down Expand Up @@ -36,6 +37,7 @@ class NetcdfFortran(AutotoolsPackage):
depends_on("netcdf-c")
depends_on("netcdf-c@4.7.4:", when="@4.5.3:") # nc_def_var_szip required
depends_on("doxygen", when="+doc", type="build")
depends_on("mpi", when="^hdf5~shared+mpi")

# The default libtool.m4 is too old to handle NAG compiler properly:
# https://github.com/Unidata/netcdf-fortran/issues/94
Expand Down Expand Up @@ -134,6 +136,26 @@ def configure_args(self):
# configuration failure, we set the following cache variable:
config_args.append("ac_cv_func_MPI_File_open=yes")

if "~shared" in netcdf_c_spec:
netcdf_libs_cmd = ["pkg-config","netcdf","--libs"]
netcdf_libs = subprocess.check_output(netcdf_libs_cmd, encoding="utf-8").strip()
config_args.append("LIBS=" + netcdf_libs)

netcdf_ldflags_cmd = ["pkg-config","netcdf","--libs-only-L"]
netcdf_ldflags = \
subprocess.check_output(netcdf_ldflags_cmd, encoding="utf8").strip()
config_args.append("LDFLAGS=" + netcdf_ldflags)

netcdf_cflags_cmd = ["pkg-config","netcdf","--cflags"]
netcdf_cflags = \
subprocess.check_output(netcdf_cflags_cmd, encoding="utf8").strip()
config_args.append("CPPFLAGS=" + netcdf_cflags)

if "~shared" in self.spec:
config_args.append("CC=%s" % self.spec["mpi"].mpicc)
config_args.append("FC=%s" % self.spec["mpi"].mpifc)
config_args.append("F77=%s" % self.spec["mpi"].mpif77)

return config_args

@run_after("configure")
Expand Down
12 changes: 12 additions & 0 deletions var/spack/repos/jcsda-emc/packages/mapl/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

import subprocess

from spack.package import *


Expand Down Expand Up @@ -62,12 +64,14 @@ class Mapl(CMakePackage):
variant("shared", default=True)
variant("debug", default=False, description="Make a debuggable version of the library")
variant("extdata2g", default=False, description="Use ExtData2G")
variant("pnetcdf", default=True, description="Use parallel netCDF")

depends_on("cmake@3.17:")
depends_on("mpi")
depends_on("hdf5")
depends_on("netcdf-c")
depends_on("netcdf-fortran")
depends_on("parallel-netcdf", when="+pnetcdf")
depends_on("esmf@8.3:", when="@2.22:")
depends_on("esmf", when="@:2.12.99")
depends_on("esmf~debug", when="~debug")
Expand Down Expand Up @@ -107,3 +111,11 @@ def cmake_args(self):
args.append(self.define("CMAKE_Fortran_FLAGS", " ".join(fflags)))

return args

def patch(self):
if "~shared" in self.spec["netcdf-c"]:
nc_pc_cmd = ["nc-config","--static","--libs"]
nc_flags = \
subprocess.check_output(nc_pc_cmd, encoding="utf8").strip()
filter_file("(target_link_libraries[^)]+PUBLIC )", \
r'\1 %s '%nc_flags, "pfio/CMakeLists.txt")