Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename dirs and make executable dir switchable to meet NCO standards #216

Closed
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,6 @@ ENDFUNCTION()
enable_testing()
include(CTest)

# Build in this subdirectory.
#add_subdirectory(src)

#####
# Configure and print the ufs-srweather-app.settings file.
#####
Expand Down Expand Up @@ -91,6 +88,10 @@ SET(host_os "${osrel}")
SET(abs_top_builddir "${CMAKE_CURRENT_BINARY_DIR}")
SET(abs_top_srcdir "${CMAKE_CURRENT_SOURCE_DIR}")

##### Executable directory name (exec_bin): exec or bin #############
SET(exec_bin exec)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems hard-coded. The PR description says it is switchable, but I don't see any option for doing that at build time. Also, what is the point of having a configurable directory name for executables? Everything should be adjustable using $CMAKE_PREFIX_PATH with its subdirs using standard software industry names. It seems to me that having a configurable directory name for the executables directory has no advantages and increases the complexity a lot. The complexity increases dramatically because now every other component in the App tree will need to support this. I need to be convinced that there is a good reason to make this configurable rather than just setting it to what it needs to be.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, I think this should look like:

set(exec_bin "exec" CACHE STRING "Executable directory name")

or perhaps

set(exec_bin "exec" CACHE INTERNAL "")

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also argue that the default should be bin not exec for the simple reason that NCO is the only place where exec would be desired. The default should be the value that the community expects.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, I am not familiar with cmake. Can you help me for this? I'll invite you to my repo.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, Now I got your point. I'll modify exec_bin soon.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, a new cmake argument '-DEMC_EXEC_DIR' was introduced to change the dir name to 'exec' for the NCO standard. This is the same approach as UFS_UTILS and UPP. Its default is 'bin'. To change the name to 'exec', we just need to add "-DEMC_EXEC_DIR=ON" to CMAKE_SETTINGS as described in devbuild.sh (ln 213).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The appropriate solution would be to provide at cmake configuration time:
-DCMAKE_INSTALL_BINDIR=exec

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also argue that the default should be bin not exec for the simple reason that NCO is the only place where exec would be desired. The default should be the value that the community expects.

That is the default that CMake assigns.

#####################################################################

SET(CC_VERSION "${CMAKE_C_COMPILER}")

# Set values for .settings file.
Expand Down Expand Up @@ -119,7 +120,7 @@ MESSAGE(${UFS-SRWEATHER-APP_SETTINGS})
# Install ufs_srweather_app.settings file into same location
# as the app.
INSTALL(FILES "${CMAKE_BINARY_DIR}/ufs_srweather_app.settings"
DESTINATION bin)
DESTINATION ${exec_bin})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use ${CMAKE_INSTALL_BINDIR}


#####
# Create 'ufs_srweather_app_meta.h' include file.
Expand All @@ -130,4 +131,4 @@ configure_file(

FILE(COPY "${CMAKE_CURRENT_BINARY_DIR}/ufs_srweather_app_meta.h" DESTINATION include)

add_subdirectory(src)
add_subdirectory(sorc)
18 changes: 11 additions & 7 deletions Externals.cfg
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
[regional_workflow]
protocol = git
repo_url = https://github.com/ufs-community/regional_workflow
#repo_url = https://github.com/ufs-community/regional_workflow
repo_url = https://github.com/chan-hoo/regional_workflow
# Specify either a branch name or a hash but not both.
#branch = develop
hash = e5d5c38
branch = feature/move_wflow_env
#hash = e5d5c38
local_path = regional_workflow
required = True

Expand All @@ -13,7 +15,7 @@ repo_url = https://github.com/ufs-community/UFS_UTILS
# Specify either a branch name or a hash but not both.
#branch = develop
hash = f30740e
local_path = src/UFS_UTILS
local_path = sorc/UFS_UTILS
required = True

[ufs-weather-model]
Expand All @@ -22,16 +24,18 @@ repo_url = https://github.com/ufs-community/ufs-weather-model
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 805421d
local_path = src/ufs-weather-model
local_path = sorc/ufs-weather-model
required = True

[UPP]
protocol = git
repo_url = https://github.com/NOAA-EMC/UPP
#repo_url = https://github.com/NOAA-EMC/UPP
repo_url = https://github.com/chan-hoo/UPP
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 4a16052
local_path = src/UPP
branch = feature/exec_bin
#hash = 4a16052
local_path = sorc/UPP
required = True

[externals_description]
Expand Down
10 changes: 5 additions & 5 deletions devbuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ usage_error () {
# default settings
LCL_PID=$$
SRC_DIR=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )" && pwd -P)
MACHINE_SETUP=${SRC_DIR}/src/UFS_UTILS/sorc/machine-setup.sh
MACHINE_SETUP=${SRC_DIR}/sorc/UFS_UTILS/sorc/machine-setup.sh
BUILD_DIR=${SRC_DIR}/build
INSTALL_DIR=${SRC_DIR}
COMPILER=""
Expand All @@ -89,12 +89,12 @@ CCPP=""
ENABLE_OPTIONS=""
DISABLE_OPTIONS=""
BUILD_TYPE="RELEASE"
BUILD_JOBS=4
BUILD_JOBS=8
CLEAN=false
CONTINUE=false
VERBOSE=false
# detect PLATFORM (MACHINE)
source ${SRC_DIR}/env/detect_machine.sh
source ${SRC_DIR}/modulefiles/detect_machine.sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of a directory named modulefiles should probably contain module files. This is confusing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, ok. I'll move it to another dir.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, this file has been moved into the 'test' dir.


# process required arguments
if [[ ("$1" == "--help") || ("$1" == "-h") ]]; then
Expand Down Expand Up @@ -161,7 +161,7 @@ if [ "${VERBOSE}" = true ] ; then
fi

# set ENV_FILE for this platform/compiler combination
ENV_FILE="${SRC_DIR}/env/build_${PLATFORM}_${COMPILER}.env"
ENV_FILE="${SRC_DIR}/modulefiles/build_${PLATFORM}_${COMPILER}.env"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're going to name this directory modulefiles then all of the *.env files should be converted to actual module files and should be loaded with, for example, module load build_hera_intel rather than sourced. This would also remove the user's shell from the equation as the module load would work regardless of the shell the user is using.

Copy link
Collaborator Author

@chan-hoo chan-hoo Feb 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, I agree. This will be my next PR. I don't want to put several topics into one PR. I'll modify the env files into the regular module files in another PR.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, if you want this change in this PR, I'll do it soon.

if [ ! -f "${ENV_FILE}" ]; then
printf "ERROR: environment file does not exist for platform/compiler\n" >&2
printf " ENV_FILE=${ENV_FILE}\n" >&2
Expand Down Expand Up @@ -231,7 +231,7 @@ fi

# source the environment file for this platform/compiler combination, then build the code
printf "... Source ENV_FILE and create BUILD directory ...\n"
module use ${SRC_DIR}/env
module use ${SRC_DIR}/modulefiles
. ${ENV_FILE}
module list
mkdir -p ${BUILD_DIR}
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
4 changes: 2 additions & 2 deletions docs/UsersGuide/README → doc/UsersGuide/README
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Steps to build and use the Sphinx documentation tool:
https://sphinxcontrib-bibtex.readthedocs.io/en/latest/quickstart.html#installation

2) Create a Sphinx documentation root directory:
% mkdir docs
% cd docs
% mkdir doc
% cd doc

3) Initialize your Sphinx project (set up an initial directory structure) using
% sphinx-quickstart
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ scripts are presented in parentheses. Some directories have been removed for br
ufs-srweather-app
├── (bin)
├── (build)
├── docs
├── doc
│ └── UsersGuide
├── (include)
├── (lib)
Expand All @@ -91,7 +91,7 @@ scripts are presented in parentheses. Some directories have been removed for br
│ ├── templates
│ └── wrappers
├── (share)
└── src
└── sorc
├── UPP
│ ├── parm
│ └── sorc
Expand Down Expand Up @@ -122,7 +122,7 @@ contents of these sub-directories are described in :numref:`Table %s <Subdirecto
+-------------------------+---------------------------------------------------------+
| **Directory Name** | **Description** |
+=========================+=========================================================+
| docs | Users' Guide Documentation |
| doc | Users' Guide Documentation |
+-------------------------+---------------------------------------------------------+
| jobs | J-job scripts launched by Rocoto |
+-------------------------+---------------------------------------------------------+
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
6 changes: 0 additions & 6 deletions env/wflow_cheyenne.env

This file was deleted.

7 changes: 0 additions & 7 deletions env/wflow_hera.env

This file was deleted.

7 changes: 0 additions & 7 deletions env/wflow_jet.env

This file was deleted.

7 changes: 0 additions & 7 deletions env/wflow_orion.env

This file was deleted.

10 changes: 0 additions & 10 deletions env/wflow_wcoss_cray.env

This file was deleted.

9 changes: 0 additions & 9 deletions env/wflow_wcoss_dell_p3.env

This file was deleted.

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
26 changes: 23 additions & 3 deletions src/CMakeLists.txt → sorc/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,30 @@
include(ExternalProject)

##### CMAKE Arguments depending on exec_bin #########################

if(${exec_bin} STREQUAL exec)
gsketefian marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the point of supporting multiple names for the executable install directory?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherwharrop-noaa, the UFS SRW App will be used as an operational model. We should change the app to meet the NCO standards for this. If you and other reviewers agree, we can use only one name as 'exec'. But I don't think this is what you want.

list(APPEND COMP_ARGS
"-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
"-DEMC_EXEC_DIR=ON"
)
else()
List(APPEND COMP_ARGS
"-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
)
endif()

##### UFS_UTILS #####################################################

ExternalProject_Add(UFS_UTILS
PREFIX ${CMAKE_CURRENT_BINARY_DIR}/UFS_UTILS
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/UFS_UTILS
INSTALL_DIR ${CMAKE_INSTALL_PREFIX}
CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
CMAKE_ARGS ${COMP_ARGS}
BUILD_ALWAYS TRUE
)

##### ufs-weather-model #############################################

if(NOT CCPP_SUITES)
set(CCPP_SUITES "FV3_CPT_v0,FV3_GFS_2017_gfdlmp,FV3_GFS_2017_gfdlmp_regional,FV3_GSD_SAR,FV3_GSD_v0,FV3_GFS_v15p2,FV3_GFS_v16,FV3_RRFS_v1beta,FV3_HRRR,FV3_RRFS_v1alpha,FV3_GFS_v15_thompson_mynn_lam3km")
endif()
Expand Down Expand Up @@ -58,14 +75,17 @@ ExternalProject_Add(ufs-weather-model
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/ufs-weather-model
INSTALL_DIR ${CMAKE_INSTALL_PREFIX}
CMAKE_ARGS ${UFS_WEATHER_MODEL_ARGS}
INSTALL_COMMAND mkdir -p ${CMAKE_INSTALL_PREFIX}/bin && cp ${CMAKE_CURRENT_BINARY_DIR}/ufs-weather-model/src/ufs-weather-model-build/ufs_model ${CMAKE_INSTALL_PREFIX}/bin/
INSTALL_COMMAND mkdir -p ${CMAKE_INSTALL_PREFIX}/${exec_bin} && cp ${CMAKE_CURRENT_BINARY_DIR}/ufs-weather-model/src/ufs-weather-model-build/ufs_model ${CMAKE_INSTALL_PREFIX}/${exec_bin}/
BUILD_ALWAYS TRUE
)

##### UPP ###########################################################

ExternalProject_Add(UPP
PREFIX ${CMAKE_CURRENT_BINARY_DIR}/UPP
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/UPP
INSTALL_DIR ${CMAKE_INSTALL_PREFIX}
CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}"
CMAKE_ARGS ${COMP_ARGS}
BUILD_ALWAYS TRUE
)