Feature/mpas model v8.1 #48
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The streamInfo argument is an instance of the MPAS_streamInfo_type derived type. With this new argument, cores can query information from the XML stream configuration file for the core to inform decisions regarding the activation or deactivation of packages.
The streamInfo argument is an instance of the MPAS_streamInfo_type derived type. With this new argument, cores can query information from the XML stream configuration file for the core to help in deciding on the stream that contains mesh information.
After a halo exchange, restrict the elements to be checked to the number of cells, edges, or vertices. The data arrays have an extra element which doesn't have valid data.
This localizes the validation code so there is only one place to modify it. This removes much redundant code. The validation code is a bit more robust, where any invalid value will fail the test. Adds debug code which will print every invalid value if HALO_EXCH_DEBUG is defined.
This merge fixes unit tests for halo exchanges in the test core, and it also refactors parts of the validation code for those tests to avoid copy-and-paste code. Previously, the halo exchange tests compared fields post-halo exchange for all field elements, including the "garbage" element, when only elements (1:nCells), (1:nVertices), or (1:nEdges) should have been compared.
…AS-Dev#1133) This merge adds the domain % streamInfo instance of an MPAS_streamInfo_type type as an argument to the setup_packages and get_mesh_stream core interface routines for all cores. Before the calls to these two core interface routines in mpas_init, the streamInfo instance has been initialized with the core's streams XML file, enabling cores to make decisions based on the streams XML file regarding packages and the stream to provide the mesh information for a core. * all/streamInfo_for_core_interfaces: Add streamInfo argument to the get_mesh_stream routine for all cores Add streamInfo argument to the setup_packages routine for all cores
In the atmosphere core's Registry.xml file, all time-invariant input fields have
been placed in a new "invariant" immutable input stream, and the "input" and
"restart" streams include the "invariant" stream.
Since the "invariant" stream is never used as the mesh input stream, the default
input file ("invariant.nc") for the stream does not need to be present in the
run directory, and the "input" and "restart" streams will expect to have all of
the invariant fields included in their files.
…core When running the streams_gen program for the atmosphere core (from src/core_atmosphere/Makefile), the command-line argument in_defaults=true is provided, so that any stream that has the "in_defaults" attribute will only be written to the default streams.atmosphere file if the value of that attribute is "true". Note that for any streams that do not specify the "in_defaults" attribute, those streams will always be written to the streams.atmosphere file.
… core The definition of the "invariant" stream in the atmosphere core now includes in_defaults="false" to prevent the stream from being written to the default streams.atmosphere file.
…e_init The atm_core_init routine now includes calls to the stream manager to attempt to read the "invariant" stream. Since the default "invariant" stream has an input_interval of "none", the call to MPAS_stream_mgr_read should effectively be a no-op. Only if the user includes the "invariant" stream in their streams.atmosphere file and sets input_interval="initial_only" will the stream be read, effectively providing a redundant read of time-invariant fields.
The atmosphere core now defines a new package, "no_invariant_stream", which is always active and is attached to the included "invariant" stream in the "input" and "restart" streams.
… mesh stream The atm_setup_packages and atm_get_mesh_stream routines now check the streams.atmosphere file to see whether it contains an "invariant" stream, and whether that invariant stream has an input_interval value of "initial_only"; if so, the "no_invariant_stream" package is deactivated and the "invariant" stream is used as the mesh stream. When the no_invariant_stream package is deactivated, neither the "input" stream nor the "restart" stream will read time-invariant fields from their files, with the expectation that these fields will have been read from the file associated with the "invariant" stream.
…1137) This merge introduces the capability to store time-invariant fields in a separate file, thereby avoiding the need to store copies of these fields in every time-varying initial conditions or restart file. Without any changes in workflow, MPAS-A will appear to a user to work as before. However, if a definition of the "invariant" input stream is added to the streams.atmosphere file at runtime, and if the input_interval for the stream is set to "initial_only", e.g., <immutable_stream name="invariant" type="input" filename_template="x1.40962.L32.invariant.nc" input_interval="initial_only" /> , the model will read all time-invariant fields from the designated file when starting up for both cold-start and restart simulations; additionally, when restart files are periodically written during a simulation, time-invariant fields will be omitted from those restart output streams. A file containing time-invariant fields can be produced while running the init_atmosphere_model program by setting config_vertical_grid = true while also setting config_met_interp = false in the namelist.init_atmosphere file for the real-data initialization case (init case 7). The invariant file can be used as input to the init_atmosphere_model program when interpolating real-data ICs with config_met_interp = true, and this same invariant file is suitable for use in the "invariant" stream in the streams.atmosphere file.
…ale-reference suite uses the nTiedtke (cu_ntiedtke) and not the tiedtke (cu_tiedtke) parameterization of deep and shallow convection. One motivation of using the cu_ntiedtke option instead of cu_tiedtke is the scale-aware capability of the nTiedtke parameterization, allowing the use of nTiedtke iwth variable-resolution meshes. -> therefore, we decided to rename the package cu_tiedtke_in to cu_ntiedtke_in.
… f_qv, f_qr, and f_qs in module_cu_tiedtke.F and module_cu_ntiedtke.F. Those three variables are not used in the parameterizations. * In ./src/core_atmosphere/physics/mpas_atmphys_driver_convection.F, removed f_qv, f_qr, and f_qs in the calls to cu_tiedtke and cu_ntiedtke.
… f_qv and f_qc in module_cu_kfeta.F. Those two variables are not used in the parameterization. * In ./src/core_atmosphere/physics/mpas_atmphys_driver_convection.F, removed f_qv and f_qc in the call to kf_eta_cps.
…added comments at the top of module.
… f_qv and f_qc in module_ra_cam.F (subroutine camrad). Those two variables are not used in the parameterization. * In ./src/core_atmosphere/physics/mpas_atmphys_driver_radiation_lw(sw).F, removed f_qv and f_qc in the calls to camrad.
…ariable f_qv which is not needed in any of the physics parameterizations.
-> in mpas_atmphys_vars.F, removed the "parameter" definition for f_qc,f_qr,f_qi,
f_qs,f_qg,f_nc,and f_ni. these variables are now simply defined as logicals.
also renamed f_qnc to f_nc, and f_qni to f_ni.
-> in mpas_atmphys_init.F, added subroutine init_physics_flags to initialize f_qc,
f_qr,f_qi,f_qs,f_nc,and f_ni.
-> in mpas_atmphys_driver_pbl.F, changed the call in mynn_bl_driver after renaming
f_qnc to f_nc, and f_qni to f_ni.
By defining the MPAS_USE_MPI_F08 macro at compile time, MPAS can now use the more modern 'mpi_f08' module for MPI. If the MPAS_USE_MPI_F08 macro is not defined, MPAS uses the 'mpi' module as prior to this commit. With the use of the 'mpi_f08' module, certain MPI types are no longer integers in Fortran, but are derived types; e.g., MPI_Comm, MPI_Request, MPI_Datatype, MPI_Info. In some instances, an integer-typed MPI type is still needed for interoperability, and the MPI standard permits this to be done through the 'mpi_val' member of all MPI derived types, e.g., MPI_Comm % mpi_val.
By trying to compile a test program that uses the 'mpi_f08' module, the top-level Makefile can now detect whether the MPI implementation provides an 'mpi_f08' module and, if so, adds a definition of the MPAS_USE_MPI_F08 macro to the CPPFLAGS used in the build.
) This merge enables MPAS to make use of the more modern mpi_f08 module introduced in MPI-3.0. The top-level Makefile now tries to compile a test program that uses mpi_f08, and if that test is successful, the macro MPAS_USE_MPI_F08 is defined in the CPPFLAGS used in the build; otherwise, MPAS will make use of the mpi module as it previously did. With the use of the mpi_f08 module, certain MPI types are no longer integers in Fortran, but are derived types; e.g., MPI_Comm, MPI_Request, MPI_Datatype, and MPI_Info. However, in some instances, an integer-typed MPI type is still needed for interoperability, and the MPI standard permits this to be done through the mpi_val member of all MPI derived types, e.g., MPI_Comm % mpi_val. * framework/mpas_use_mpi_f08: Add logic in top-level Makefile to detect mpi_f08 module availability Allow MPAS to use either of 'mpi_f08' or 'mpi' modules
-> renamed module mpas_atmphys_driver_lsm_shared.F to mpas_atmphys_lsm_shared.F.
-> in mpas_atmphys_driver_lsm.F, now use mpas_atmphys_lsm_shared to call subroutine
correct_tsk_over_seaice. removed subroutine correct_tsk_over_seaice at the bottom
of the module.
-> corrected Makefile accordingly.
…eres The methods previously employed in the mpas_sphere_angle function for computing the side lengths of a spherical triangle were only applicable to unit-radius spheres. This commit generalizes the computation of these side lengths so that the mpas_sphere_angle function can be used to compute spherical angles on spheres of arbitrary radius.
JCSDA-internal/MPAS-Model repository. Note the number of halo layers impacts the number of cells which will be updated by this routine: The first halo layer will update the owned 'edge' cells, where 'edge' cells are adjacent to ghost cells. The second halo layer will update owned cells which are adjacent to the 'edge' cells. The third halo layer will update owned cells which are adjacent to the cells updated by the seconds halo layer, etc.
…gh the local variables needed in the calculation of t2m, th2m, and q2 prior to updating all LSM variables (see subroutine lsm_to_MPAS).
…he call to sfcdiags from subroutine driver_lsm to subroutine lsm_to_MPAS. This allows t2m, th2m, and q2 to be correctly computed over seaice points.
MPI implementations not compiled with mpi_f08 enabled still pass the check. Using OpenMPI 4.0.0 and gfortran 11.4.1 compiling without mpi_f08 features the MPI wrapper will still link and supply mpi_f08 (-lmpi_usempif08) despite not being valid or usable. This causes the `use mpi_f08` and some features to compile fine, but other mpi_f08 features to not exist and fail compilation.
This follows the same logic as before with the added check that a libpnetcdf.* file exists in the checked directory. The change in logic will now account for situations where PNETCDF is set to a directory containing both lib/ and lib64/ directories and libpnetcdf.* exists in just the lib/ folder. Should libpnetcdf.* exist in both directories, the lib64/ still maintains precedence.
This commit removes some debugging log writes around calls to sfclayrev in the driver_sfclayer subroutine. These writes led to messages like the following appearing in the log file for every model timestep: --- enter subroutine sfclayrev: --- end subroutine sfclayrev: --- enter subroutine sfclayrev seaice: --- end subroutine sfclayrev seaice:
…(PR MPAS-Dev#1206) This merge removes some debugging log writes around calls to sfclayrev in the driver_sfclayer subroutine. These writes led to messages like the following appearing in the log file for every model timestep: --- enter subroutine sfclayrev: --- end subroutine sfclayrev: --- enter subroutine sfclayrev seaice: --- end subroutine sfclayrev seaice: * atmosphere/remove_sfclayrev_prints: Remove debugging prints around calls to sfclayrev in driver_sfclayer
…) arrays This commit adds a new routine to the dmpar module for broadcasing real(kind=R4KIND) arrays. The new mpas_dmpar_bcast_real4s routine mirrors the mpas_dmpar_bcast_reals routine, with the dummy array argument being of R4KIND rather than RKIND kind.
…sics.F The mpas_atmphys_init_microphysics module contains a routine, table_ccnAct, that broadcasts an R4KIND real array, tnccn_act, from the Thompson microphysics scheme. This broadcast was previously performed by a call to mpas_dmpar_bcast_reals through the macro DM_BCAST_MACRO. However, because the mpas_dmpar_bcast_reals routine broadcasts arrays of kind RKIND, when MPAS-A is compiled with double-precision reals, the mpas_dmpar_bcast_reals routine can no longer be used with the tnccn_act array. With this commit, the DM_BCAST_MACRO in mpas_atmphys_init_microphysics.F has been defined to mpas_dmpar_bcast_real4s, which broadcasts arrays of R4KIND reals. This works for both single- and double-precision builds of MPAS-A, since the kind type of the tnccn_act array is independent of the MPAS-A build precision.
…#1207) This merge adds a new mpas_dmpar_bcast_real4s routine to the mpas_dmpar module for broadcasting real(kind=R4KIND) arrays. The mpas_dmpar_bcast_real4s routine mirrors the mpas_dmpar_bcast_reals routine, with the dummy array argument being of R4KIND rather than RKIND kind. * framework/bcast_real4s: Add mpas_dmpar_bcast_real4s routine for broadcasting real(kind=R4KIND) arrays
…ev#1208) This merge corrects double-precision build failures of mpas_atmphys_init_microphysics.F in MPAS-A v8.2.0. The mpas_atmphys_init_microphysics module contains a routine, table_ccnAct, that broadcasts an R4KIND real array, tnccn_act, from the Thompson microphysics scheme. This broadcast was previously performed by a call to mpas_dmpar_bcast_reals through the macro DM_BCAST_MACRO. However, because the mpas_dmpar_bcast_reals routine broadcasts arrays of kind RKIND, when MPAS-A is compiled with double-precision reals, the mpas_dmpar_bcast_reals routine can no longer be used with the tnccn_act array. With this merge, the DM_BCAST_MACRO in mpas_atmphys_init_microphysics.F has been defined to mpas_dmpar_bcast_real4s, which broadcasts arrays of R4KIND reals. This works for both single- and double-precision builds of MPAS-A, since the kind type of the tnccn_act array is independent of the MPAS-A build precision. * atmosphere/fix_r8_builds: Correct double-precision build failures of mpas_atmphys_init_microphysics.F
…l instances when spaces were used instead of tabs (lines 41-43, and lines 104, 108, 111, and 114.)
This commit restores CMake support for the atmosphere core, which broke due to changes made prior to the version 8.2 release. The breaking changes included moving the physics_mmm directory to its own git repository. This is accounted for in CMake by using FetchContent to clone the correct version of the physics_mmm source code directory based on the MPAS model git version. Additionally, this commit builds the noahmp library that was added to the code base. Previously, this library only supported being built with a hardcoded GNU Makefile. This commit also accounts for the new registry preprocessing arguments that make the registry processing core-agnostic. These changes re-enable a seamless build process for the atmosphere core and improve modularity and maintainability.
This commit restores CMake support for the atmosphere core, which broke due to changes made prior to the version 8.2 release. The breaking changes were a result of the addition of mpas_init_atm_thompson_aerosols.F to the init_atmosphere core without updating the CMakeLists.txt. To fix this problem, this file was appended to the list of source files in the init_atmosphere core's CMakeLists.txt.
…F, corrected the calculation of the height Above Ground Level (AGL) prior to computing radar in subroutine compute_radar_reflectivity.
) This merge fixes the CMake builds for both the init_atmosphere and atmosphere cores. Prior to releasing MPAS v8.2.0, several changes necessitated updates to the GNU Makefiles. These changes included: * Inclusion of the Noah-MP library * Renaming of several existing files * Removing the physics_mmm directory from the source code and moving it to its own GitHub repository * Adding a source file to the init_atmosphere core * Modifying how the Registry files are processed The CMake build files were not updated accordingly, resulting in the loss of CMake support in MPAS v8.2.0. This merge includes the necessary updates to the CMake files, enabling MPAS-A to be configured and built with CMake. These updates ensure that the latest version of MPAS-A can be used with MPAS-JEDI and allow MPAS-Model to be compiled using both GNU and Intel compilation suites. * atmosphere/fix_cmake: Restore CMake support for building the init_atmosphere core Restore CMake support for building the atmosphere core
…PAS-Dev#1212) This merge corrects several whitespace (indentation) inconsistencies in src/core_atmosphere/physics/physics_wrf/Makefile. In the original Makefile for the physics_wrf directory, there were several instances when indentation used spaces instead of tabs, and this merge replaces spaces with tabs in those instances. * NSF-MPAS/physics_Makefile_tabs: * In ./src/core_atmosphere/physics/physics_wrf/Makefile, corrected all instances...
…1202) This merge adds the use of more entities from the 'mpi_f08' module in the mpi_f08 test program in the top-level Makefile in order to more robustly detect whether an MPI implementation provides a working 'mpi_f08' module. Anecdotally, some MPI implementations (e.g., OpenMPI 4.0.0 with the GNU 11.4.1 compilers) permit the existing mpi_f08 test program to build, but these implementations nonetheless don't properly provide all mpi_f08 features. By using additional entities from the 'mpi_f08' module in the test program, these MPI implementations can be identified as not providing complete mpi_f08 support. Note that the additional entities from the 'mpi_f08' module that are introduced into the test program in this merge are used in MPAS code anyway, and so there is no harm in verifying that they are provided at the test program stage.
As the netcdf library logic mirrors the pnetcdf one, it would suffer from the same issues noted in 0ca4208 about lib/lib64 checking. This updates the corresponding issue in the netcdf library detection before it becomes an issue.
-> removed package bl_mynn_in from packages included for nc and tend_nc. the
array declaration for nc, nifa, nwfa, tend_nc, tend_nifa, and tend_nwfa
now includes the same packages.
-> in ./physics/mpas_atmphys_interface.F, added arrays nifa_p, and nwfa_p for
the PBL option "bl_mynn".
-> in ./physics/mpas_atmphys_todynamics.F, only updates tend_scalars(index_nc,
tend_scalars(index_nifa, and tend_scalars(index_nwfa due to PBL processes
when the microp_scheme option is "mp_thompson_aerosols".
These changes are needed to run the MYNN PBL scheme without the aerosol-aware
option of the Thompson cloud microphysics scheme.
) This merge improves the detection of the locations of the netCDF and pNetCDF libraries. Prior to this merge, the top-level Makefile contained logic to check for the existence of directories $(NETCDF)/lib and $(NETCDF)/lib64, and similarly for $(PNETCDF)/lib and $(PNETCDF)/lib64. This logic didn't actually include checks on the presence of libraries within those directories. Since the presence of a lib64/ directory was checked after the check for a lib/ directory, if, say, the netCDF library were installed in lib/ and not in lib64/, the result of the Makefile logic would be an incorrect netCDF path of $(NETCDF)/lib64 if a lib64/ directory was also present. With this merge, the Makefile logic checks for the existence of libnetcdf.* or libpnetcdf.* files within the lib/ and lib64/ directories before selecting a directory for the path to netCDF or PnetCDF. * check-pnetcdf-exists: Check for the libnetcdf library existence similar to pnetcdf Check for libpnetcdf library existence rather than just directory
…ize.o Add the object dependencies in the makefile for mpas_atmphys_lsm_noahmpfinalize.o specific to the objects created under core_physics, i.e. any use statement that reference modules compiled as part of the core_physics target. Modules originating from NoahMP and framework are ignored as they should be handled separately before the core_physics target is started.
When building MPAS-A as a dycore, all physics-related components are disabled. An example scenario of this is for use with CAM/CAM-SIMA. However, the MPAS-A registry file in version 8.2.0 contains the following regressions, which cause the dycore-only build to break: * The MPAS-A registry file introduced a new stream, `da_state`, which unconditionally includes variables that exist only when physics are enabled. * The MPAS-A registry file unconditionally includes a new registry file from Noah-MP, which is applicable only when physics are enabled. Fix MPAS-A dycore-only build by guarding the above regions with the `DO_PHYSICS` macro.
…1 (PR MPAS-Dev#1204) This merge adds a missing dependency on mpas_atmphys_var.o for mpas_atmphys_lsm_noahmpfinalize.o in the MPAS-A physics Makefile. * mpas_atmphys_lsm_noahmpfinalize-deps: Add core_physic-specific dependencies to mpas_atmphys_lsm_noahmpfinalize.o
…Dev#1221) This merge fixes dycore-only (i.e., no-physics) builds for both stand-alone applications and when MPAS-A is used as a dycore in CAM/CAM-SIMA. Prior to this merge there were two issues in the atmosphere core's Registry.xml file that caused dynamics-only builds to fail: * With the MPAS v8.2.0 release, the MPAS-A registry file introduced a new stream, da_state, which unconditionally included variables that exist only when physics are enabled. * Also beginning with the MPAS v8.2.0 release, the Registry.xml file unconditionally included a new registry file from Noah-MP, which is applicable only when physics are enabled. This merge fixes MPAS-A dycore-only builds by guarding the above regions in the Registry.xml file with the DO_PHYSICS macro. * stg/fix-dycore-only-build: Fix MPAS-A dycore-only build
…S-Dev#1213) This merge corrects the calculation of height above ground level (AGL) that is used in the computation of the 1-km simulated radar reflectivity field in mpas_atmphys_driver_microphysics.F. Specifically, the fixes in this merge affect the refl10cm_1km and refl10cm_1km_max fields. * NSF-MPAS/physics_big_fix_zd: * In ./src/core_atmosphere//physics/mpas_atmphys_driver_microphysics.F, corrected...
…anged print statements "enter/end subroutine init_atm_gocart" with "enter/end subroutine init_atm_thompson_aerosols.
…, removed extra deallocate for variable tao5503d_p in subroutine deallocate_radiation_sw.F.
MPAS-Dev#1215) This merge fixes an issue that prevented the use of the MYNN PBL scheme (e.g., through the selection of the 'convection_permitting' suite) without the use of the aerosol-aware Thompson cloud microphysics scheme.
…PAS-Dev#1224) This merge corrects the subroutine name used in two calls to mpas_log_write in mpas_init_atm_thompson_aerosols.F, where "init_atm_gocart" should have been "init_atm_thompson_aerosols". Note that the affected mpas_log_write calls are commented-out and so their messages don't appear in log files by default.
…PAS-Dev#1226) This merge removes an extra deallocation of the 'taod5503d_p' array in the deallocate_radiation_sw() routine in mpas_atmphys_driver_radiation_sw.F. The taod5503d_p array is only allocated when running the aerosol-aware Thompson cloud microphysics, and the extra deallocation that is removed in this merge only deallocated taod5503d_p if it was allocated.
This merge addresses several issues in the MPAS-Atmosphere model and in the MPAS infrastructure. Specific changes in this merge include: * Improved detection of an 'mpi_f08' module (PR MPAS-Dev#1202), as well as improved detection of netCDF and PnetCDF library paths (PR MPAS-Dev#1203), in the top-level Makefile. * The addition of a missing dependency in the physics Makefile to correct parallel build issues (PR MPAS-Dev#1204). * Fixes to the CMake build files used by MPAS-JEDI (PR MPAS-Dev#1205). * Fixes to double-precision builds of MPAS-Atmosphere (PR MPAS-Dev#1207, PR MPAS-Dev#1208). * Correction of the calculation of height AGL used in the computation of 1-km radar reflectivity fields (PR MPAS-Dev#1213). * Correction of an issue that prevented the MYNN PBL scheme from being used without also using the Thompson aerosol-aware microphysics (PR MPAS-Dev#1215). * Fixes to allow MPAS-Atmosphere to be built without physics (i.e., dynamics- only builds) (PR MPAS-Dev#1221). * Various code cleanup and minor corrections (PR MPAS-Dev#1206, PR MPAS-Dev#1212, PR MPAS-Dev#1224, PR MPAS-Dev#1226).
|
Based on the test results in https://github.com/JCSDA-internal/mpas-jedi/pull/990 - it appears this PR is no longer needed? Unless you wanted to merge these changes anyways... although I'm under the impression that we will be archiving this repo once the change is in place. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR was created to meet the CI requirements for MPAS-JEDI, which now depends on MPAS-Model v8.2.1 from the official MPAS-Dev repository. To fulfill the CI dependency requirements, I created this branch from the MPAS-Dev/MPAS-Model v8.2.1 tag. This branch is solely intended to satisfy CI dependencies and will not be merged into the JCSDA-internal fork.