Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to move SRW app towards NCO complaince #337

Merged
merged 16 commits into from
Aug 30, 2022
Merged

Changes to move SRW app towards NCO complaince #337

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
7c07edc
Build to exec directory instead of bin.
danielabdi-noaa Aug 14, 2022
ac1d194
Rename src to sorc
danielabdi-noaa Aug 14, 2022
643437f
Add separate building of rrfs components.
danielabdi-noaa Aug 14, 2022
2bed9be
Temporarily change bin directory from "exec" to "bin"
danielabdi-noaa Aug 14, 2022
3eb3d65
Remove --rrfs option from test/build, print packages to build.
danielabdi-noaa Aug 14, 2022
5572495
Update hash of regional_workflow.
danielabdi-noaa Aug 14, 2022
68236cf
Minor changes.
danielabdi-noaa Aug 14, 2022
88c80ea
Set default list for direct cmake build path.
danielabdi-noaa Aug 14, 2022
4e604f1
Separte clean,build and install steps.
danielabdi-noaa Aug 15, 2022
b8cc65a
Bug fix remove vs clean
danielabdi-noaa Aug 16, 2022
bada8af
Remove custom install step cause it is same as all target anyway.
danielabdi-noaa Aug 16, 2022
207b6e1
Update for EMC_EXEC_DIR removal from UPP.
danielabdi-noaa Aug 24, 2022
3f5e7ef
Use modulefiles of component instead of top-level modulefile.
danielabdi-noaa Aug 29, 2022
552c78f
Make modulefile name search flexible.
danielabdi-noaa Aug 30, 2022
1ed233a
Remove GSI dependency of rrfs_utl assuming gsi is already built.
danielabdi-noaa Aug 30, 2022
1b51621
Fallback to SRW modules when we cant find modules of sub-component.
danielabdi-noaa Aug 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,14 @@ FUNCTION(is_enabled feature ret_val)
ENDIF(${feature})
ENDFUNCTION()

# User options
option(ENABLE_RRFS "Build RRFS components" OFF)

# Turn on testing.
enable_testing()
include(CTest)

# Build in this subdirectory.
#add_subdirectory(src)
# Default bin directory
if(NOT DEFINED CMAKE_INSTALL_BINDIR)
set(CMAKE_INSTALL_BINDIR bin)
endif()

#####
# Configure and print the ufs-srweather-app.settings file.
Expand Down Expand Up @@ -122,7 +121,7 @@ MESSAGE(${UFS-SRWEATHER-APP_SETTINGS})
# Install ufs_srweather_app.settings file into same location
# as the app.
INSTALL(FILES "${CMAKE_BINARY_DIR}/ufs_srweather_app.settings"
DESTINATION bin)
DESTINATION ${CMAKE_INSTALL_BINDIR})

#####
# Create 'ufs_srweather_app_meta.h' include file.
Expand All @@ -133,4 +132,7 @@ configure_file(

FILE(COPY "${CMAKE_CURRENT_BINARY_DIR}/ufs_srweather_app_meta.h" DESTINATION include)

add_subdirectory(src)
add_custom_target(mbuild DEPENDS sorc/pbuild)
add_custom_target(minstall DEPENDS sorc/pinstall)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the magic that separates the build and install steps?

Are the names of the targets arbitrary? I suppose I'm curious why mbuild and minstall instead of just build and install.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but the cause of the problem was the external projects add. You would expect a make all would just build the binaries and not install but default behavior of ExternalProject_Add is non-intiutive -- it builds and installs on a make all. So I added STEP_TARGETS build install to separate them and then connect to a custom target at the top level. I think future CMake version could potentially simplify this. I used mbuild and minstall because I couldn't use "install" as target since it is a reserved keyword. "build" was fine though but changed it to be consistent.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the custom install target because there was already one anyway, so we have a build target instead of mbuild now. Still couldn't find a solution to prevent make all from installing the project...


add_subdirectory(sorc)
12 changes: 6 additions & 6 deletions Externals.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ protocol = git
repo_url = https://github.com/ufs-community/regional_workflow
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 3a2cb82
hash = b17ac12
local_path = regional_workflow
required = True

Expand All @@ -13,7 +13,7 @@ repo_url = https://github.com/ufs-community/UFS_UTILS
# Specify either a branch name or a hash but not both.
#branch = develop
hash = c5cfcf4
local_path = src/UFS_UTILS
local_path = sorc/UFS_UTILS
required = True

[ufs-weather-model]
Expand All @@ -22,7 +22,7 @@ repo_url = https://github.com/ufs-community/ufs-weather-model
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 65cf401
local_path = src/ufs-weather-model
local_path = sorc/ufs-weather-model
required = True

[UPP]
Expand All @@ -31,7 +31,7 @@ repo_url = https://github.com/NOAA-EMC/UPP
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 8767929
local_path = src/UPP
local_path = sorc/UPP
required = True

[GSI]
Expand All @@ -40,7 +40,7 @@ repo_url = https://github.com/NOAA-EMC/GSI.git
# Specify either a branch name or a hash but not both.
#branch = develop
hash = ad84f17
local_path = src/gsi
local_path = sorc/gsi
externals = None
required = True

Expand All @@ -50,7 +50,7 @@ repo_url = https://github.com/NOAA-GSL/rrfs_utl.git
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 8d21f25
local_path = src/rrfs_utl
local_path = sorc/rrfs_utl
required = True

[externals_description]
Expand Down
107 changes: 88 additions & 19 deletions devbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# usage instructions
usage () {
cat << EOF_USAGE
Usage: $0 --platform=PLATFORM [OPTIONS]...
Usage: $0 --platform=PLATFORM [OPTIONS] ... [TARGETS]

OPTIONS
-h, --help
Expand All @@ -19,8 +19,6 @@ OPTIONS
(e.g. ATM | ATMW | S2S | S2SW)
--ccpp="CCPP_SUITE1,CCPP_SUITE2..."
CCPP suites (CCPP_SUITES) to include in build; delimited with ','
--rrfs
build rrfs system components: GSI, rrfs_utl
--enable-options="OPTION1,OPTION2,..."
enable ufs-weather-model options; delimited with ','
(e.g. 32BIT | INLINE_POST | UFS_GOCART | MOM6 | CICE6 | WW3 | CMEPS)
Expand All @@ -29,12 +27,20 @@ OPTIONS
(e.g. 32BIT | INLINE_POST | UFS_GOCART | MOM6 | CICE6 | WW3 | CMEPS)
--continue
continue with existing build
--clean
--remove
removes existing build; overrides --continue
--clean
does a "make clean"
--build
does a "make"
--install
does a "make install"
--build-dir=BUILD_DIR
build directory
--install-dir=INSTALL_DIR
installation prefix
--bin-dir=BIN_DIR
installation binary directory name ("bin" or "exec")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these the only two options? Could this be set arbitrarily?

Copy link
Collaborator Author

@danielabdi-noaa danielabdi-noaa Aug 16, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no. UPP and UFS_UTILS do not use CMAKE_INSTALL_BINDIR as the destination target, instead they set to either "bin" or "exec" where the latter is used if an "EMC_EXEC_DIR" variable is turned on. But once these are fixed, we should be able to install to any bin directory name.

--build-type=BUILD_TYPE
build type; defaults to RELEASE
(e.g. DEBUG | RELEASE | RELWITHDEBINFO)
Expand All @@ -43,6 +49,11 @@ OPTIONS
-v, --verbose
build with verbose output

TARGETS
default = builds the default list of apps (also not passing any target does the same)
all = builds all apps
Or any combinations of (ufs, ufs_utils, upp, gsi, rrfs_utils)

NOTE: This script is for internal developer use only;
See User's Guide for detailed build instructions

Expand All @@ -57,17 +68,23 @@ Settings:
SRW_DIR=${SRW_DIR}
BUILD_DIR=${BUILD_DIR}
INSTALL_DIR=${INSTALL_DIR}
BIN_DIR=${BIN_DIR}
PLATFORM=${PLATFORM}
COMPILER=${COMPILER}
APP=${APPLICATION}
CCPP=${CCPP_SUITES}
ENABLE_OPTIONS=${ENABLE_OPTIONS}
DISABLE_OPTIONS=${DISABLE_OPTIONS}
CLEAN=${CLEAN}
REMOVE=${REMOVE}
CONTINUE=${CONTINUE}
BUILD_TYPE=${BUILD_TYPE}
BUILD_JOBS=${BUILD_JOBS}
VERBOSE=${VERBOSE}
BUILD_UFS=${BUILD_UFS}
BUILD_UFS_UTILS=${BUILD_UFS_UTILS}
BUILD_UPP=${BUILD_UPP}
BUILD_GSI=${BUILD_GSI}
BUILD_RRFS_UTILS=${BUILD_RRFS_UTILS}

EOF_SETTINGS
}
Expand All @@ -85,18 +102,31 @@ SRW_DIR=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )" && pwd -P)
MACHINE_SETUP=${SRW_DIR}/src/UFS_UTILS/sorc/machine-setup.sh
BUILD_DIR="${SRW_DIR}/build"
INSTALL_DIR=${SRW_DIR}
BIN_DIR="bin" #change this to "exec" for NCO mode later
COMPILER=""
APPLICATION=""
CCPP_SUITES=""
RRFS="off"
ENABLE_OPTIONS=""
DISABLE_OPTIONS=""
BUILD_TYPE="RELEASE"
BUILD_JOBS=4
CLEAN=false
REMOVE=false
CONTINUE=false
VERBOSE=false

# Turn off all apps to build and choose default later
DEFAULT_BUILD=true
BUILD_UFS="off"
BUILD_UFS_UTILS="off"
BUILD_UPP="off"
BUILD_GSI="off"
BUILD_RRFS_UTILS="off"

# Make options
CLEAN=false
BUILD=false
INSTALL=false

# process required arguments
if [[ ("$1" == "--help") || ("$1" == "-h") ]]; then
usage
Expand All @@ -115,32 +145,53 @@ while :; do
--app|--app=|-a|-a=) usage_error "$1 requires argument." ;;
--ccpp=?*) CCPP_SUITES=${1#*=} ;;
--ccpp|--ccpp=) usage_error "$1 requires argument." ;;
--rrfs) RRFS="on" ;;
--rrfs=*) usage_error "$1 argument ignored." ;;
--enable-options=?*) ENABLE_OPTIONS=${1#*=} ;;
--enable-options|--enable-options=) usage_error "$1 requires argument." ;;
--disable-options=?*) DISABLE_OPTIONS=${1#*=} ;;
--disable-options|--disable-options=) usage_error "$1 requires argument." ;;
--clean) CLEAN=true ;;
--clean=?*|--clean=) usage_error "$1 argument ignored." ;;
--remove) REMOVE=true ;;
--remove=?*|--remove=) usage_error "$1 argument ignored." ;;
--continue) CONTINUE=true ;;
--continue=?*|--continue=) usage_error "$1 argument ignored." ;;
--clean) CLEAN=true ;;
--build) BUILD=true ;;
--install) INSTALL=true ;;
--build-dir=?*) BUILD_DIR=${1#*=} ;;
--build-dir|--build-dir=) usage_error "$1 requires argument." ;;
--install-dir=?*) INSTALL_DIR=${1#*=} ;;
--install-dir|--install-dir=) usage_error "$1 requires argument." ;;
--bin-dir=?*) BIN_DIR=${1#*=} ;;
--bin-dir|--bin-dir=) usage_error "$1 requires argument." ;;
--build-type=?*) BUILD_TYPE=${1#*=} ;;
--build-type|--build-type=) usage_error "$1 requires argument." ;;
--build-jobs=?*) BUILD_JOBS=$((${1#*=})) ;;
--build-jobs|--build-jobs=) usage_error "$1 requires argument." ;;
--verbose|-v) VERBOSE=true ;;
--verbose=?*|--verbose=) usage_error "$1 argument ignored." ;;
# targets
default) ;;
all) DEFAULT_BUILD=false; BUILD_UFS="on";
BUILD_UFS_UTILS="on"; BUILD_UPP="on";
BUILD_GSI="on"; BUILD_RRFS_UTILS="on";;
ufs) DEFAULT_BUILD=false; BUILD_UFS="on" ;;
ufs_utils) DEFAULT_BUILD=false; BUILD_UFS_UTILS="on" ;;
upp) DEFAULT_BUILD=false; BUILD_UPP="on" ;;
gsi) DEFAULT_BUILD=false; BUILD_GSI="on" ;;
rrfs_utils) DEFAULT_BUILD=false; BUILD_RRFS_UTILS="on" ;;
# unknown
-?*|?*) usage_error "Unknown option $1" ;;
*) break
esac
shift
done

# choose default apps to build
if [ "${DEFAULT_BUILD}" = true ]; then
BUILD_UFS="on"
BUILD_UFS_UTILS="on"
BUILD_UPP="on"
fi

# Ensure uppercase / lowercase ============================================
APPLICATION="${APPLICATION^^}"
PLATFORM="${PLATFORM,,}"
Expand Down Expand Up @@ -197,7 +248,7 @@ fi
printf "MODULE_FILE=${MODULE_FILE}\n" >&2

# if build directory already exists then exit
if [ "${CLEAN}" = true ]; then
if [ "${REMOVE}" = true ]; then
printf "Remove build directory\n"
printf " BUILD_DIR=${BUILD_DIR}\n\n"
rm -rf ${BUILD_DIR}
Expand Down Expand Up @@ -231,17 +282,22 @@ else
fi

# cmake settings
CMAKE_SETTINGS="-DCMAKE_INSTALL_PREFIX=${INSTALL_DIR}"
CMAKE_SETTINGS="${CMAKE_SETTINGS} -DCMAKE_BUILD_TYPE=${BUILD_TYPE}"
CMAKE_SETTINGS="\
-DCMAKE_BUILD_TYPE=${BUILD_TYPE}\
-DCMAKE_INSTALL_PREFIX=${INSTALL_DIR}\
-DCMAKE_INSTALL_BINDIR=${BIN_DIR}\
-DBUILD_UFS=${BUILD_UFS}\
-DBUILD_UFS_UTILS=${BUILD_UFS_UTILS}\
-DBUILD_UPP=${BUILD_UPP}\
-DBUILD_GSI=${BUILD_GSI}\
-DBUILD_RRFS_UTILS=${BUILD_RRFS_UTILS}"

if [ ! -z "${APPLICATION}" ]; then
CMAKE_SETTINGS="${CMAKE_SETTINGS} -DAPP=${APPLICATION}"
fi
if [ ! -z "${CCPP_SUITES}" ]; then
CMAKE_SETTINGS="${CMAKE_SETTINGS} -DCCPP_SUITES=${CCPP_SUITES}"
fi
if [ ! -z "${RRFS}" ]; then
CMAKE_SETTINGS="${CMAKE_SETTINGS} -DENABLE_RRFS=${RRFS}"
fi
if [ ! -z "${ENABLE_OPTIONS}" ]; then
CMAKE_SETTINGS="${CMAKE_SETTINGS} -DENABLE_OPTIONS=${ENABLE_OPTIONS}"
fi
Expand All @@ -265,9 +321,22 @@ module load ${MODULE_FILE}
module list
mkdir -p ${BUILD_DIR}
cd ${BUILD_DIR}

printf "... Generate CMAKE configuration ...\n"
cmake ${SRW_DIR} ${CMAKE_SETTINGS} 2>&1 | tee log.cmake
printf "... Compile executables ...\n"
make ${MAKE_SETTINGS} 2>&1 | tee log.make

if [ "${CLEAN}" = true ]; then
printf "... Clean executables ...\n"
make ${MAKE_SETTINGS} clean 2>&1 | tee log.make
elif [ "${BUILD}" = true ]; then
printf "... Compile executables ...\n"
make ${MAKE_SETTINGS} mbuild 2>&1 | tee log.make
elif [ "${INSTALL}" = true ]; then
printf "... Install executables ...\n"
make ${MAKE_SETTINGS} minstall 2>&1 | tee log.make
else
printf "... Compile and install executables ...\n"
make ${MAKE_SETTINGS} 2>&1 | tee log.make
fi

exit 0
Loading