Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update DA cmake build #112

Closed
RussTreadon-NOAA opened this issue Feb 15, 2021 · 1 comment · Fixed by #327
Closed

Update DA cmake build #112

RussTreadon-NOAA opened this issue Feb 15, 2021 · 1 comment · Fixed by #327

Comments

@RussTreadon-NOAA
Copy link
Contributor

This issue is opened to document updates to the cmake build used by DA apps in NOAA-EMC/GSI. This development is being done in https://github.com/aerorahul/GSI/tree/feature/cmake-update.
@RussTreadon-NOAA
Copy link
Contributor Author

As discussed in hpc-stack issue #149, it was discovered that one of the libraries in CORE_LIBRARIES used to build the GSI interacts with -lm in such a way to alter global_gsi.x results. hpc-stack @ 3ceffb1 is configured to link global_gsi.x using /usr/lib64/libm.so with hpc-stack/1.1.0. With this change hpc-stack @ 3ceffb1 generates b4b identical results with respect to the NOAA-EMC/GSI master

The extra /usr/lib64/libm.so line was removed from a working copy of hpc-stack @ 3ceffb1 and global_gsi.x was built. global_gsi.x was built from cmake-update @ 07fb370. Both global_gsi.x executables were run for several cases using input from NCO's v16 parallel. In all cases the two executable generate b4b identical analysis increment files.

@MichaelLueken MichaelLueken linked a pull request May 12, 2022 that will close this issue
GitHub Issue NOAA-EMC/GSI#112. A refactor of CMake build framework. #327
Merged
MichaelLueken added a commit to MichaelLueken/GSI that referenced this issue May 18, 2022
@MichaelLueken
GitHub Issue NOAA-EMC#112. A refactor of CMake build framework.
79d227a
MichaelLueken added a commit that referenced this issue May 18, 2022
@aerorahul @RussTreadon-NOAA
@MichaelLueken @christopherwharrop-noaa
GitHub Issue #112. A refactor of CMake build framework. (#327)
85ea172 
* WIP: cleanup top-level CMakeLists.txt

* clean up ncdiag cmake build

* build global_gsi.x. regional requires something from wrflib

* wcoss_d build

* hierarchy of compiler flags

* Update Intel Fortran flags

* order of includes matter

* install built library and executable for GSI

* Issue #112:  update intel fortran flags, add intel c flags, replace fv3gfs_ncio with ncio module, replace bufr_4_DA with bufr_d

* updates for GNU v10

* add missing files in gsi_files.cmake

* build works with GNU for GSI library and apps.  Install creates package config

* gsi is its own project now

* build GSI and EnKF GFS flavors

* ncdiag, gsi and enkf can be now built independently

* build all (except Monitoring) utilities

* update logic when GSI and EnKF are not found for utilities

* build mods for Orion

* make messages identify their origin

* promote Release instead of Production

* add -DNDEBUG flags for gsi and enkf

* revert whitespace differences that were unintended

* remove obsolete cmake/Modules directory

* remove top-level cmake/ directory

* add host specific flags to module files

* make packages more robust and handle transitive libraries once and done.

* add options for building different components.

* add utilities that are used but did not have CMake builds

* Revert "remove obsolete cmake/Modules directory"

This reverts commit 91fb0ac.

* ncdiag is a requirement since the stubs are obsolete

* add FindMKL.cmake.  Remove finding ncdiag from gsi as a dependency.

* Build changes for WCOSS Dell

* Make necessary changes to allow feature/cmake-refactor to run regression tests: (#1)

Changes from @MichaelLueken-NOAA on setting up offline regression testing.

* Build conventional monitoring utilities.  There is a lot of repeated code in these and could benefit from making a library rather than copy/paste of the same code for each utility.

* Build Radiance and Ozone monitoring utilities.  There is a lot of repeated code in these and could benefit from making a library rather than copy/paste of the same code for each utility.

* retain trailing white spaces to minimize diffs

* add place-holder for regression testing

* remove dependency on NDATE from regression tests.  Use Unix date instead.

* reinstate regression testing via ctest.

* Fix duplicate definition of MPI variables (#3)

* Fix duplicate definition of MPI variables
* Replace "include mpif.h" with "use mpi"

* minor update to EnKF/CMakeLists.txt

* reinstate regression tests w/ ctest

* remove stale cmake/Modules

* remove broken Debug CPP definitions from ncdiag library

* Build with GNU and CMAKE_BUILD_TYPE=Debug

* make DEBUG Intel flags consistent

* allow users to provide CMAKE_OPTS without having to edit the script

* make util build options consistent

* Fix logical comparison bug caught by Gnu build (#4)

* bugfix for cheyenne

* fix logical comparison

* build cleanly

* fix improper character array allocation in fv3reg

* cleanup the build script to allow substantial user control

* add wrapper for NCO build script.
Address GFS adaptation for NCO requirement of "pruning"
Add BUILD_UTIL_ALL option for convenience

* update INSTALL.md to add BUILD_UTIL_ALL option

* trap pruning error

* fix the error with machine_id for cray and dell

* fix the error with machine_id for cray and dell

* update cmake version in orion modulefile

* update ncio to v1.1.0

* consolidate common modulefiles in a modulefile.ProdGSI.common

* add intel and gnu modulefiles for Cheyenne from Chris Harrop

* retain only common modules in modulefile.ProdGSI.common;

* add missing #%Module in the common modulefile

* revert back to ncio/1.0.0

* Fixes for Cheyenne and Jet (#5)

* Fix duplicate MPI definitions in EFSOI exposed by Cheyenne build

Set CC, FC, CXX explicitly for Cheyenne GNU to avoid
use of incorrect MPI module.

Update w3emc to version 2.9.2.

* Fix MPI API call bug that passed mpi_status type instead of a status variable

* Revert version of w3emc and update cmake version for Jet

* remove build.comgsi as it is unlikely to work with the current refactor. Use the `build.sh` instead

* Keeping up with #369, where only some of the modulefiles were updated to lua.

* convert remaining modulefiles to lua.

* modulefile.ProdGSI.common.lua was buggy.  a source code change slipped through via one of the many merged PRs

* remove prod_util from common and load in the platform itself

* since we are changing modulefiles, make their names short and consistent with other applications

* wcoss_dell_p3 cmake is from hpc-stack. Fix that.
update detect_machine.sh and module-setup.sh for wcoss2.
be smart in auto-detection.
fix warnings in source codes.

* protect against accidental commits of binaries

* explicitly use calls from MPI module

Co-authored-by: russ.treadon <Russ.Treadon@noaa.gov>
Co-authored-by: Rahul Mahajan <rmahajan@ice.local>
Co-authored-by: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Co-authored-by: Christopher Harrop <35781497+christopherwharrop-noaa@users.noreply.github.com>
MichaelLueken pushed a commit that referenced this issue May 18, 2022
@aerorahul @MichaelLueken
GitHub Issue #112. A refactor of CMake build framework. (#327)
7146a0b
MichaelLueken pushed a commit that referenced this issue May 27, 2022
@aerorahul @MichaelLueken
GitHub Issue #112. A refactor of CMake build framework. (#327)
ddee392
EdwardSafford-NOAA pushed a commit to EdwardSafford-NOAA/GSI that referenced this issue May 27, 2022
@aerorahul @MichaelLueken
GitHub Issue NOAA-EMC#112. A refactor of CMake build framework. (NOAA…
bc38de4 
…-EMC#327)
aerorahul added a commit to NOAA-EMC/GSI-Monitor that referenced this issue Jun 2, 2022
@aerorahul @MichaelLueken
GitHub Issue NOAA-EMC/GSI#112. A refactor of CMake build framework. (…
d5f91df 
…#327)
AndrewEichmann-NOAA pushed a commit to AndrewEichmann-NOAA/GSI that referenced this issue Jun 6, 2022
@aerorahul @MichaelLueken
GitHub Issue NOAA-EMC#112. A refactor of CMake build framework. (NOAA…
cd5bde7 
…-EMC#327)
aerorahul added a commit to NOAA-EMC/GSI-ncdiag that referenced this issue Jun 13, 2022
@aerorahul @MichaelLueken
GitHub Issue NOAA-EMC/GSI#112. A refactor of CMake build framework. (…
f27aee9 
…#327)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/GSI that referenced this issue Jun 21, 2022
@RussTreadon-NOAA
GitHub Issue NOAA-EMC#266. remove global_cycle from DA jobs and scripts
d19b2b0 
commit 112ca82
Merge: 257f6eb dd1bac5
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue Jun 21 09:56:10 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit 257f6eb
Merge: 9c48261 ad84f17
Author: RussTreadon-NOAA <russ.treadon@noaa.gov>
Date:   Mon Jun 13 14:52:28 2022 +0000

    Merge branch 'develop' into feature/rm_sfcanl

commit 9c48261
Merge: dc62da7 eb220ca
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Thu May 26 05:43:20 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit eb220ca
Author: michael.lueken <Michael.Lueken@noaa.gov>
Date:   Wed May 25 16:04:25 2022 +0000

    GitHub Issue NOAA-EMC#382.  Add GSI fix file changes for v16.3

commit 8c22b33
Merge: a90b632 cf4b323
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Wed May 25 15:56:49 2022 +0000

    Merge pull request NOAA-EMC#374 from ADCollard/IRBugFix

    GitHub Issue NOAA-EMC#371. Minor bug fixes to IR processing

commit dc62da7
Merge: b7afecf a90b632
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Fri May 20 12:10:17 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit a90b632
Merge: 524527b 889cce5
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Fri May 20 16:08:11 2022 +0000

    Merge pull request NOAA-EMC#386 from HaixiaLiu-NOAA/master

    GitHub Issue NOAA-EMC#385. Add seviri_m09 to Radiance_Monitor util

commit 889cce5
Author: Haixia.Liu <Haixia.Liu@noaa.gov>
Date:   Fri May 20 04:27:21 2022 +0000

    GitHub Issue NOAA-EMC#385. Add seviri_m09 to Radiance_Monitor util

commit b7afecf
Merge: 58b52c6 524527b
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Thu May 19 12:46:19 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit 524527b
Merge: 9946c69 002dc87
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Thu May 19 16:08:48 2022 +0000

    Merge pull request NOAA-EMC#379 from AndrewEichmann-NOAA/EXP-efso_fv3_issue_378

    Github issue NOAA-EMC#378: refactoring of EFSOI scripts and compiled code (EFSOI and EFSOI forecast scripts)

commit 9946c69
Merge: bcc0358 ccfda16
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Thu May 19 16:05:20 2022 +0000

    Merge pull request NOAA-EMC#381 from emilyhcliu/feature/gmi

    GitHub Issue NOAA-EMC#341. Add GMI for radmon

commit bcc0358
Merge: 7146a0b 7a4fa9b
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Thu May 19 16:02:55 2022 +0000

    Merge pull request NOAA-EMC#384 from christopherwharrop-noaa/feature/update-cheyenne-hpc-stack

    GitHub Issue NOAA-EMC#383 Update hpc-stack in Cheyenne modulefiles for GNU and Intel compilers.

commit 7a4fa9b
Author: Christopher Harrop <Christopher.W.Harrop@noaa.gov>
Date:   Thu May 19 09:23:33 2022 -0600

    GitHub Issue NOAA-EMC#383

    Update hpc-stack in Cheyenne modulefiles for GNU and Intel compilers.

    A small workaround was necessary for the GNU modulefile to avoid
    a dependency issue with the GNU version of hpc-stack.

commit ccfda16
Author: Emily Liu <emily.liu@noaa.gov>
Date:   Thu May 19 02:20:09 2022 +0000

    Add GMI for radmon

commit 58b52c6
Merge: c4c860b 7146a0b
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Wed May 18 14:52:48 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit 7146a0b
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date:   Wed May 18 11:57:08 2022 -0400

    GitHub Issue NOAA-EMC#112.  A refactor of CMake build framework. (NOAA-EMC#327)

commit cf4b323
Author: Andrew Collard <andrew.collard@noaa.gov>
Date:   Fri Apr 29 21:34:22 2022 +0000

    GitHub Issue NOAA-EMC#371: Bug fixes to IR processing

commit 002dc87
Author: AndrewEichmann-NOAA <Andrew.Eichmann@noaa.gov>
Date:   Thu May 12 15:02:40 2022 -0500

    Removed extraneous lines from EFSOI script, and extraneous
    exgdas_efsoi_fcst.sh script

commit c4c860b
Merge: c9ac824 34294c1
Author: RussTreadon-NOAA <russ.treadon@noaa.gov>
Date:   Mon May 9 16:46:28 2022 +0000

    Merge branch 'develop' into feature/rm_sfcanl

commit 34294c1
Merge: ad2a3c0 20cf887
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Mon May 9 16:26:12 2022 +0000

    Merge pull request NOAA-EMC#376 from EdwardSafford-NOAA/develop

    GitHub Issue NOAA-EMC#373.  Compress RadMon data files.

commit ad2a3c0
Merge: e7f8c88 845b360
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Mon May 9 16:24:29 2022 +0000

    Merge pull request NOAA-EMC#369 from MichaelLueken-NOAA/feature/wcoss2_port

    GitHub Issue NOAA-EMC#368.  WCOSS2 port.

commit 20cf887
Author: Edward.Safford <edward.safford@noaa.gov>
Date:   Thu May 5 12:04:30 2022 +0000

    Github issue NOAA-EMC#373.  Compress RadMon data files.  Completes NOAA-EMC#373.

commit 845b360
Author: Michael Lueken <michael.lueken@noaa.gov>
Date:   Fri Apr 29 11:00:52 2022 +0000

    GitHub Issue NOAA-EMC#368.  WCOSS2 port.

commit c9ac824
Merge: e577941 e7f8c88
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue May 3 10:42:43 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit e7f8c88
Merge: 586c2df 0a5daea
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Tue May 3 10:08:34 2022 -0400

    Merge pull request NOAA-EMC#372 from RussTreadon-NOAA/feature/interp

    GitHub Issue NOAA-EMC#370: use nf90_inq_varid to set integer addresses for lon and lat variables

commit 0a5daea
Author: RussTreadon-NOAA <russ.treadon@noaa.gov>
Date:   Fri Apr 29 21:36:35 2022 +0000

    GitHub Issue NOAA-EMC/GSI NOAA-EMC#370: use nf90_inq_varid to set integer addresses for lon and lat variables

commit e577941
Merge: be550bf 586c2df
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue May 3 09:52:08 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit 586c2df
Merge: 87fe77d f16e852
Author: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com>
Date:   Tue May 3 09:46:33 2022 -0400

    Merge pull request NOAA-EMC#366 from EdwardSafford-NOAA/develop

    GitHub Issue NOAA-EMC#310 and NOAA-EMC#367. Correct RadMon's mean correction time series plots and update html generation.

commit f16e852
Author: edward.safford <edward.safford@noaa.gov>
Date:   Fri Apr 29 14:39:22 2022 +0000

    Github issue NOAA-EMC#310.  Correct mean correction time series plots.  Fixes NOAA-EMC#310.

commit be550bf
Merge: c0a5204 114ed31
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Mon May 2 06:25:23 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit c0a5204
Merge: 8b376d1 498896b
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Mon Apr 25 13:22:30 2022 -0400

    Merge branch 'NOAA-EMC:develop' into feature/rm_sfcanl

commit 8b376d1
Merge: 91a8077 0c6548e
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Wed Apr 20 13:53:51 2022 -0400

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit 91a8077
Merge: dec3299 7201778
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Wed Apr 13 10:27:36 2022 -0400

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit dec3299
Merge: 6204f98 5936a23
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue Apr 5 10:49:24 2022 -0400

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit 6204f98
Merge: f7a1495 affe4ed
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Thu Mar 31 12:38:40 2022 -0400

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit f7a1495
Merge: b592666 ef237ce
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Thu Mar 24 08:57:46 2022 -0400

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit b592666
Merge: d0f7c20 47ee70b
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Thu Mar 10 09:13:14 2022 -0500

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit d0f7c20
Merge: ed94193 828c0cf
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Wed Mar 2 10:35:21 2022 -0500

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit ed94193
Merge: c63bd77 963b7e5
Author: RussTreadon-NOAA <russ.treadon@noaa.gov>
Date:   Tue Mar 1 16:04:48 2022 +0000

    Merge branch 'master' at 963b7e5 into feature/rm_sfcanl

    Conflicts:
    	scripts/exglobal_atmos_analysis.sh

commit c63bd77
Merge: 39912e1 0444258
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue Feb 15 06:06:10 2022 -0500

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit 39912e1
Merge: f0161c8 9e765da
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue Jan 11 06:40:58 2022 -0500

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit f0161c8
Merge: a75d6ed a62dec6
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date:   Tue Dec 14 11:40:31 2021 -0500

    Merge branch 'NOAA-EMC:master' into feature/rm_sfcanl

commit a75d6ed
Author: russ.treadon <Russ.Treadon@noaa.gov>
Date:   Tue Dec 14 14:38:57 2021 +0000

    GitHub Issue NOAA-EMC#266:  add DOGAUSFCANL capability back to JGLOBAL_ATMOS_ANALYSIS_CALC and exglobal_atmos_analysis_calc.sh

commit 9422758
Author: russ.treadon <Russ.Treadon@noaa.gov>
Date:   Mon Dec 13 14:39:14 2021 +0000

    GitHub Issue NOAA-EMC#266:  remove DOGCYCLE and DOGAUSFCANL scripting and variables from j-jobs and scripts

     * jobs/JGDAS_ATMOS_ANALYSIS_DIAG - remove DOGCYCLE and DOGAUSFCANL
     * jobs/JGLOBAL_ATMOS_ANALYSIS - remove DOGCYCLE and DOGAUSFCANL
     * jobs/JGLOBAL_ATMOS_ANALYSIS_CALC - remove DOGCYCLE and DOGAUSFCANL
     * scripts/exglobal_atmos_analysis.sh - remove DOGCYCLE scripting and DOGAUSFCANL variables
     * scripts/exglobal_atmos_analysis_calc.sh - remove DOGAUSFCANL scripting
aerorahul added a commit to NOAA-EMC/GSI-utils that referenced this issue Jun 30, 2022
@aerorahul @MichaelLueken
GitHub Issue NOAA-EMC/GSI#112. A refactor of CMake build framework. (…
e8f92bd 
…#327)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

GitHub Issue NOAA-EMC/GSI#112. A refactor of CMake build framework.
1 participant
@RussTreadon-NOAA