Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UFS-dev PR#42 #92

Merged
merged 11 commits into from
Apr 12, 2023
Merged

UFS-dev PR#42 #92

merged 11 commits into from
Apr 12, 2023

Conversation

grantfirl
Copy link
Collaborator

Identical to ufs-community#1642

Also contains changes from ufs-community#1656.

Contains changes from #91 until it is merged and this branch is updated.

BrianCurtis-NOAA and others added 3 commits March 21, 2023 09:42
* aqm init_concentrations=false

* ATMAQ tests are 32BIT
* use the latest postxconfig-NT-fv3lam.txt to run ifi

* rename postxconfig-NT-fv3lam-ifi.txt to postxconfig-NT-fv3lam-new.txt since it is, in fact, the new postxconfig-NT-fv3lam.txt

* add lightning threat indexes to rrfs 13km tests

* pass w to model & fixes to ccpp/physics changes

* FV3: turn off lightning_threat by default since it allocates wgrs and passes w

* do not output ltg*_max variables since they're all zeroes anyway because the 13km run cannot make enough graupel

* FV3: missing `active = (lightning_threat_indices_enabled)` for wgrd

* enable ltg*_max variables in diag_table_hrrr

* alternative POSTAPP for C96 versions of regional configurations

* rap & hrrr variants use POSTAPP=fv3lam_global

* new tests for ifi on acorn

* fix out-of-bounds access in upp calslr_roebbr

* fix another out-of-bounds access and move a message to stdout
@grantfirl grantfirl mentioned this pull request Mar 31, 2023
@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: gnu
Job: RT
[RT] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230406151507/ufs-weather-model
[RT] Error: Test control_stochy 002 failed in check_result failed
[RT] Error: Test control_stochy 002 failed in run_test failed
[RT] Error: Test control_ras 003 failed in check_result failed
[RT] Error: Test control_ras 003 failed in run_test failed
[RT] Error: Test control_p8 004 failed in check_result failed
[RT] Error: Test control_p8 004 failed in run_test failed
[RT] Error: Test rap_control 005 failed in check_result failed
[RT] Error: Test rap_control 005 failed in run_test failed
[RT] Error: Test rap_decomp 006 failed in check_result failed
[RT] Error: Test rap_decomp 006 failed in run_test failed
[RT] Error: Test rap_2threads 007 failed in check_result failed
[RT] Error: Test rap_2threads 007 failed in run_test failed
[RT] Error: Test rap_sfcdiff 009 failed in check_result failed
[RT] Error: Test rap_sfcdiff 009 failed in run_test failed
[RT] Error: Test rap_sfcdiff_decomp 010 failed in check_result failed
[RT] Error: Test rap_sfcdiff_decomp 010 failed in run_test failed
[RT] Error: Test hrrr_control 012 failed in check_result failed
[RT] Error: Test hrrr_control 012 failed in run_test failed
[RT] Error: Test hrrr_control_2threads 013 failed in check_result failed
[RT] Error: Test hrrr_control_2threads 013 failed in run_test failed
[RT] Error: Test hrrr_control_decomp 014 failed in check_result failed
[RT] Error: Test hrrr_control_decomp 014 failed in run_test failed
[RT] Error: Test rrfs_v1beta 016 failed in check_result failed
[RT] Error: Test rrfs_v1beta 016 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 017 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 017 failed in run_test failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 018 failed in check_result failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 018 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 019 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 019 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 020 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 020 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 032 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 032 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 033 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 033 failed in run_test failed
[RT] Error: Test rap_control_dyn32_phy32 035 failed in check_result failed
[RT] Error: Test rap_control_dyn32_phy32 035 failed in run_test failed
[RT] Error: Test hrrr_control_dyn32_phy32 036 failed in check_result failed
[RT] Error: Test hrrr_control_dyn32_phy32 036 failed in run_test failed
[RT] Error: Test rap_2threads_dyn32_phy32 037 failed in check_result failed
[RT] Error: Test rap_2threads_dyn32_phy32 037 failed in run_test failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 038 failed in check_result failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 038 failed in run_test failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 039 failed in check_result failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 039 failed in run_test failed
[RT] Error: Test rap_control_dyn64_phy32 042 failed in check_result failed
[RT] Error: Test rap_control_dyn64_phy32 042 failed in run_test failed
Please make changes and add the following label back: cheyenne-gnu-RT

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: hera
Compiler: gnu
Job: RT
[RT] Repo location: /scratch1/BMC/gmtb/RT/auto_RT/Pull_Requests/1298047098/20230406211510/ufs-weather-model
[RT] Error: Test control_stochy 002 failed in check_result failed
[RT] Error: Test control_stochy 002 failed in run_test failed
[RT] Error: Test control_ras 003 failed in check_result failed
[RT] Error: Test control_ras 003 failed in run_test failed
[RT] Error: Test control_p8 004 failed in check_result failed
[RT] Error: Test control_p8 004 failed in run_test failed
[RT] Error: Test rap_control 005 failed in check_result failed
[RT] Error: Test rap_control 005 failed in run_test failed
[RT] Error: Test rap_decomp 006 failed in check_result failed
[RT] Error: Test rap_decomp 006 failed in run_test failed
[RT] Error: Test rap_2threads 007 failed in check_result failed
[RT] Error: Test rap_2threads 007 failed in run_test failed
[RT] Error: Test rap_sfcdiff 009 failed in check_result failed
[RT] Error: Test rap_sfcdiff 009 failed in run_test failed
[RT] Error: Test rap_sfcdiff_decomp 010 failed in check_result failed
[RT] Error: Test rap_sfcdiff_decomp 010 failed in run_test failed
[RT] Error: Test hrrr_control 012 failed in check_result failed
[RT] Error: Test hrrr_control 012 failed in run_test failed
[RT] Error: Test hrrr_control_2threads 013 failed in check_result failed
[RT] Error: Test hrrr_control_2threads 013 failed in run_test failed
[RT] Error: Test hrrr_control_decomp 014 failed in check_result failed
[RT] Error: Test hrrr_control_decomp 014 failed in run_test failed
[RT] Error: Test rrfs_v1beta 016 failed in check_result failed
[RT] Error: Test rrfs_v1beta 016 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 017 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 017 failed in run_test failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 018 failed in check_result failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 018 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 019 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 019 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 020 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 020 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 032 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 032 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 033 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 033 failed in run_test failed
[RT] Error: Test rap_control_dyn32_phy32 035 failed in check_result failed
[RT] Error: Test rap_control_dyn32_phy32 035 failed in run_test failed
[RT] Error: Test hrrr_control_dyn32_phy32 036 failed in check_result failed
[RT] Error: Test hrrr_control_dyn32_phy32 036 failed in run_test failed
[RT] Error: Test rap_2threads_dyn32_phy32 037 failed in check_result failed
[RT] Error: Test rap_2threads_dyn32_phy32 037 failed in run_test failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 038 failed in check_result failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 038 failed in run_test failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 039 failed in check_result failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 039 failed in run_test failed
[RT] Error: Test rap_control_dyn64_phy32 042 failed in check_result failed
[RT] Error: Test rap_control_dyn64_phy32 042 failed in run_test failed
Please make changes and add the following label back: hera-gnu-RT

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: intel
Job: RT
[RT] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230406155428/ufs-weather-model
[RT] Error: Test control_latlon 021 failed in check_result failed
[RT] Error: Test control_latlon 021 failed in run_test failed
[RT] Error: Test control_wrtGauss_netcdf_parallel 022 failed in check_result failed
[RT] Error: Test control_wrtGauss_netcdf_parallel 022 failed in run_test failed
[RT] Error: Test control_c192 024 failed in check_result failed
[RT] Error: Test control_c192 024 failed in run_test failed
[RT] Error: Test control_c384 025 failed in check_result failed
[RT] Error: Test control_c384 025 failed in run_test failed
[RT] Error: Test control_c384gdas 026 failed in check_result failed
[RT] Error: Test control_c384gdas 026 failed in run_test failed
[RT] Error: Test control_stochy 027 failed in check_result failed
[RT] Error: Test control_stochy 027 failed in run_test failed
[RT] Error: Test control_lndp 029 failed in check_result failed
[RT] Error: Test control_lndp 029 failed in run_test failed
[RT] Error: Test control_iovr4 030 failed in check_result failed
[RT] Error: Test control_iovr4 030 failed in run_test failed
[RT] Error: Test control_iovr5 031 failed in check_result failed
[RT] Error: Test control_iovr5 031 failed in run_test failed
[RT] Error: Test control_p8 032 failed in check_result failed
[RT] Error: Test control_p8 032 failed in run_test failed
[RT] Error: Test control_p8_lndp 033 failed in check_result failed
[RT] Error: Test control_p8_lndp 033 failed in run_test failed
[RT] Error: Test control_decomp_p8 035 failed in check_result failed
[RT] Error: Test control_decomp_p8 035 failed in run_test failed
[RT] Error: Test control_2threads_p8 036 failed in check_result failed
[RT] Error: Test control_2threads_p8 036 failed in run_test failed
[RT] Error: Test control_p8_rrtmgp 037 failed in check_result failed
[RT] Error: Test control_p8_rrtmgp 037 failed in run_test failed
[RT] Error: Test merra2_thompson 038 failed in check_result failed
[RT] Error: Test merra2_thompson 038 failed in run_test failed
[RT] Error: Test regional_control 039 failed in check_result failed
[RT] Error: Test regional_control 039 failed in run_test failed
[RT] Error: Test regional_decomp 041 failed in check_result failed
[RT] Error: Test regional_decomp 041 failed in run_test failed
[RT] Error: Test regional_2threads 042 failed in check_result failed
[RT] Error: Test regional_2threads 042 failed in run_test failed
[RT] Error: Test regional_2dwrtdecomp 045 failed in check_result failed
[RT] Error: Test regional_2dwrtdecomp 045 failed in run_test failed
[RT] Error: Test regional_wofs 046 failed in check_result failed
[RT] Error: Test regional_wofs 046 failed in run_test failed
[RT] Error: Test rap_control 047 failed in check_result failed
[RT] Error: Test rap_control 047 failed in run_test failed
[RT] Error: Test regional_spp_sppt_shum_skeb 048 failed in check_result failed
[RT] Error: Test regional_spp_sppt_shum_skeb 048 failed in run_test failed
[RT] Error: Test rap_decomp 049 failed in check_result failed
[RT] Error: Test rap_decomp 049 failed in run_test failed
[RT] Error: Test rap_2threads 050 failed in check_result failed
[RT] Error: Test rap_2threads 050 failed in run_test failed
[RT] Error: Test rap_sfcdiff 052 failed in check_result failed
[RT] Error: Test rap_sfcdiff 052 failed in run_test failed
[RT] Error: Test rap_sfcdiff_decomp 053 failed in check_result failed
[RT] Error: Test rap_sfcdiff_decomp 053 failed in run_test failed
[RT] Error: Test hrrr_control 055 failed in check_result failed
[RT] Error: Test hrrr_control 055 failed in run_test failed
[RT] Error: Test hrrr_control_decomp 056 failed in check_result failed
[RT] Error: Test hrrr_control_decomp 056 failed in run_test failed
[RT] Error: Test hrrr_control_2threads 057 failed in check_result failed
[RT] Error: Test hrrr_control_2threads 057 failed in run_test failed
[RT] Error: Test rrfs_v1beta 059 failed in check_result failed
[RT] Error: Test rrfs_v1beta 059 failed in run_test failed
[RT] Error: Test rrfs_v1nssl 060 failed in check_result failed
[RT] Error: Test rrfs_v1nssl 060 failed in run_test failed
[RT] Error: Test rrfs_v1nssl_nohailnoccn 061 failed in check_result failed
[RT] Error: Test rrfs_v1nssl_nohailnoccn 061 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 062 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 062 failed in run_test failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 063 failed in check_result failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 063 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 064 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 064 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_2threads 065 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_2threads 065 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 066 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 066 failed in run_test failed
[RT] Error: Test control_csawmg 067 failed in check_result failed
[RT] Error: Test control_csawmg 067 failed in run_test failed
[RT] Error: Test control_csawmgt 068 failed in check_result failed
[RT] Error: Test control_csawmgt 068 failed in run_test failed
[RT] Error: Test control_ras 069 failed in check_result failed
[RT] Error: Test control_ras 069 failed in run_test failed
[RT] Error: Test control_p8_faster 071 failed in check_result failed
[RT] Error: Test control_p8_faster 071 failed in run_test failed
[RT] Error: Test regional_control_faster 072 failed in check_result failed
[RT] Error: Test regional_control_faster 072 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 073 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 073 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 074 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 074 failed in run_test failed
[RT] Error: Test regional_spp_sppt_shum_skeb_dyn32_phy32 099 failed in check_result failed
[RT] Error: Test regional_spp_sppt_shum_skeb_dyn32_phy32 099 failed in run_test failed
[RT] Error: Test rap_control_dyn32_phy32 100 failed in check_result failed
[RT] Error: Test rap_control_dyn32_phy32 100 failed in run_test failed
[RT] Error: Test hrrr_control_dyn32_phy32 101 failed in check_result failed
[RT] Error: Test hrrr_control_dyn32_phy32 101 failed in run_test failed
[RT] Error: Test rap_2threads_dyn32_phy32 102 failed in check_result failed
[RT] Error: Test rap_2threads_dyn32_phy32 102 failed in run_test failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 103 failed in check_result failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 103 failed in run_test failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 104 failed in check_result failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 104 failed in run_test failed
[RT] Error: Test rap_control_dyn64_phy32 107 failed in check_result failed
[RT] Error: Test rap_control_dyn64_phy32 107 failed in run_test failed
[RT] Error: Test hafs_regional_atm 111 failed in check_result failed
[RT] Error: Test hafs_regional_atm 111 failed in run_test failed
[RT] Error: Test hafs_global_multiple_4nests_atm 119 failed in check_result failed
[RT] Error: Test hafs_global_multiple_4nests_atm 119 failed in run_test failed
[RT] Error: Test hafs_regional_specified_moving_1nest_atm 120 failed in check_result failed
[RT] Error: Test hafs_regional_specified_moving_1nest_atm 120 failed in run_test failed
[RT] Error: Test control_atmwav 147 failed in check_result failed
[RT] Error: Test control_atmwav 147 failed in run_test failed
[RT] Error: Test atmaero_control_p8 148 failed in check_result failed
[RT] Error: Test atmaero_control_p8 148 failed in run_test failed
[RT] Error: Test atmaero_control_p8_rad 149 failed in check_result failed
[RT] Error: Test atmaero_control_p8_rad 149 failed in run_test failed
[RT] Error: Test atmaero_control_p8_rad_micro 150 failed in check_result failed
[RT] Error: Test atmaero_control_p8_rad_micro 150 failed in run_test failed
[RT] Error: Test regional_atmaq 151 failed in check_result failed
[RT] Error: Test regional_atmaq 151 failed in run_test failed
[RT] Error: Test regional_atmaq_faster 152 failed in check_result failed
[RT] Error: Test regional_atmaq_faster 152 failed in run_test failed
[RT] Error: Test compile_008 failed in run_compile failed
Please make changes and add the following label back: cheyenne-intel-RT

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: hera
Compiler: intel
Job: RT
[RT] Repo location: /scratch1/BMC/gmtb/RT/auto_RT/Pull_Requests/1298047098/20230406223023/ufs-weather-model
[RT] Error: Test cpld_bmark_p8 011 failed in check_result failed
[RT] Error: Test cpld_bmark_p8 011 failed in run_test failed
[RT] Error: Test control_latlon 024 failed in check_result failed
[RT] Error: Test control_latlon 024 failed in run_test failed
[RT] Error: Test control_wrtGauss_netcdf_parallel 025 failed in check_result failed
[RT] Error: Test control_wrtGauss_netcdf_parallel 025 failed in run_test failed
[RT] Error: Test control_c192 027 failed in check_result failed
[RT] Error: Test control_c192 027 failed in run_test failed
[RT] Error: Test control_c384 028 failed in check_result failed
[RT] Error: Test control_c384 028 failed in run_test failed
[RT] Error: Test control_c384gdas 029 failed in check_result failed
[RT] Error: Test control_c384gdas 029 failed in run_test failed
[RT] Error: Test control_stochy 030 failed in check_result failed
[RT] Error: Test control_stochy 030 failed in run_test failed
[RT] Error: Test control_lndp 032 failed in check_result failed
[RT] Error: Test control_lndp 032 failed in run_test failed
[RT] Error: Test control_iovr4 033 failed in check_result failed
[RT] Error: Test control_iovr4 033 failed in run_test failed
[RT] Error: Test control_iovr5 034 failed in check_result failed
[RT] Error: Test control_iovr5 034 failed in run_test failed
[RT] Error: Test control_p8 035 failed in check_result failed
[RT] Error: Test control_p8 035 failed in run_test failed
[RT] Error: Test control_p8_lndp 036 failed in check_result failed
[RT] Error: Test control_p8_lndp 036 failed in run_test failed
[RT] Error: Test control_decomp_p8 038 failed in check_result failed
[RT] Error: Test control_decomp_p8 038 failed in run_test failed
[RT] Error: Test control_2threads_p8 039 failed in check_result failed
[RT] Error: Test control_2threads_p8 039 failed in run_test failed
[RT] Error: Test control_p8_rrtmgp 040 failed in check_result failed
[RT] Error: Test control_p8_rrtmgp 040 failed in run_test failed
[RT] Error: Test merra2_thompson 041 failed in check_result failed
[RT] Error: Test merra2_thompson 041 failed in run_test failed
[RT] Error: Test regional_control 042 failed in check_result failed
[RT] Error: Test regional_control 042 failed in run_test failed
[RT] Error: Test regional_decomp 044 failed in check_result failed
[RT] Error: Test regional_decomp 044 failed in run_test failed
[RT] Error: Test regional_2threads 045 failed in check_result failed
[RT] Error: Test regional_2threads 045 failed in run_test failed
[RT] Error: Test regional_2dwrtdecomp 048 failed in check_result failed
[RT] Error: Test regional_2dwrtdecomp 048 failed in run_test failed
[RT] Error: Test regional_wofs 049 failed in check_result failed
[RT] Error: Test regional_wofs 049 failed in run_test failed
[RT] Error: Test rap_control 050 failed in check_result failed
[RT] Error: Test rap_control 050 failed in run_test failed
[RT] Error: Test regional_spp_sppt_shum_skeb 051 failed in check_result failed
[RT] Error: Test regional_spp_sppt_shum_skeb 051 failed in run_test failed
[RT] Error: Test rap_decomp 052 failed in check_result failed
[RT] Error: Test rap_decomp 052 failed in run_test failed
[RT] Error: Test rap_2threads 053 failed in check_result failed
[RT] Error: Test rap_2threads 053 failed in run_test failed
[RT] Error: Test rap_sfcdiff 055 failed in check_result failed
[RT] Error: Test rap_sfcdiff 055 failed in run_test failed
[RT] Error: Test rap_sfcdiff_decomp 056 failed in check_result failed
[RT] Error: Test rap_sfcdiff_decomp 056 failed in run_test failed
[RT] Error: Test hrrr_control 058 failed in check_result failed
[RT] Error: Test hrrr_control 058 failed in run_test failed
[RT] Error: Test hrrr_control_decomp 059 failed in check_result failed
[RT] Error: Test hrrr_control_decomp 059 failed in run_test failed
[RT] Error: Test hrrr_control_2threads 060 failed in check_result failed
[RT] Error: Test hrrr_control_2threads 060 failed in run_test failed
[RT] Error: Test rrfs_v1beta 062 failed in check_result failed
[RT] Error: Test rrfs_v1beta 062 failed in run_test failed
[RT] Error: Test rrfs_v1nssl 063 failed in check_result failed
[RT] Error: Test rrfs_v1nssl 063 failed in run_test failed
[RT] Error: Test rrfs_v1nssl_nohailnoccn 064 failed in check_result failed
[RT] Error: Test rrfs_v1nssl_nohailnoccn 064 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 065 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm 065 failed in run_test failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 066 failed in check_result failed
[RT] Error: Test rrfs_smoke_conus13km_hrrr_warm 066 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 067 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm 067 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_2threads 068 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_2threads 068 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 069 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_2threads 069 failed in run_test failed
[RT] Error: Test control_csawmg 070 failed in check_result failed
[RT] Error: Test control_csawmg 070 failed in run_test failed
[RT] Error: Test control_csawmgt 071 failed in check_result failed
[RT] Error: Test control_csawmgt 071 failed in run_test failed
[RT] Error: Test control_ras 072 failed in check_result failed
[RT] Error: Test control_ras 072 failed in run_test failed
[RT] Error: Test control_p8_faster 074 failed in check_result failed
[RT] Error: Test control_p8_faster 074 failed in run_test failed
[RT] Error: Test regional_control_faster 075 failed in check_result failed
[RT] Error: Test regional_control_faster 075 failed in run_test failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 076 failed in check_result failed
[RT] Error: Test rrfs_conus13km_hrrr_warm_debug 076 failed in run_test failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 077 failed in check_result failed
[RT] Error: Test rrfs_conus13km_radar_tten_warm_debug 077 failed in run_test failed
[RT] Error: Test regional_spp_sppt_shum_skeb_dyn32_phy32 102 failed in check_result failed
[RT] Error: Test regional_spp_sppt_shum_skeb_dyn32_phy32 102 failed in run_test failed
[RT] Error: Test rap_control_dyn32_phy32 103 failed in check_result failed
[RT] Error: Test rap_control_dyn32_phy32 103 failed in run_test failed
[RT] Error: Test hrrr_control_dyn32_phy32 104 failed in check_result failed
[RT] Error: Test hrrr_control_dyn32_phy32 104 failed in run_test failed
[RT] Error: Test rap_2threads_dyn32_phy32 105 failed in check_result failed
[RT] Error: Test rap_2threads_dyn32_phy32 105 failed in run_test failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 106 failed in check_result failed
[RT] Error: Test hrrr_control_2threads_dyn32_phy32 106 failed in run_test failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 107 failed in check_result failed
[RT] Error: Test hrrr_control_decomp_dyn32_phy32 107 failed in run_test failed
[RT] Error: Test rap_control_dyn64_phy32 110 failed in check_result failed
[RT] Error: Test rap_control_dyn64_phy32 110 failed in run_test failed
[RT] Error: Test hafs_regional_atm 114 failed in check_result failed
[RT] Error: Test hafs_regional_atm 114 failed in run_test failed
[RT] Error: Test hafs_global_multiple_4nests_atm 122 failed in check_result failed
[RT] Error: Test hafs_global_multiple_4nests_atm 122 failed in run_test failed
[RT] Error: Test hafs_regional_specified_moving_1nest_atm 123 failed in check_result failed
[RT] Error: Test hafs_regional_specified_moving_1nest_atm 123 failed in run_test failed
[RT] Error: Test control_atmwav 150 failed in check_result failed
[RT] Error: Test control_atmwav 150 failed in run_test failed
[RT] Error: Test atmaero_control_p8 151 failed in check_result failed
[RT] Error: Test atmaero_control_p8 151 failed in run_test failed
[RT] Error: Test atmaero_control_p8_rad 152 failed in check_result failed
[RT] Error: Test atmaero_control_p8_rad 152 failed in run_test failed
[RT] Error: Test atmaero_control_p8_rad_micro 153 failed in check_result failed
[RT] Error: Test atmaero_control_p8_rad_micro 153 failed in run_test failed
[RT] Error: Test regional_atmaq 154 failed in check_result failed
[RT] Error: Test regional_atmaq 154 failed in run_test failed
[RT] Error: Test regional_atmaq_debug 155 failed in check_result failed
[RT] Error: Test regional_atmaq_debug 155 failed in run_test failed
[RT] Error: Test regional_atmaq_faster 156 failed in check_result failed
[RT] Error: Test regional_atmaq_faster 156 failed in run_test failed
Please make changes and add the following label back: hera-intel-RT

@grantfirl
Copy link
Collaborator Author

@dustinswales I looked through the associated develop PRs and didn't find much in terms of which RTs are expected to fail, and I haven't heard back from Sam for clues. I think that we have to depend on the develop PRs passing and getting merged and that they were OK with whatever baselines changed so as not to delay this.

on-behalf-of @NCAR <dswales@ucar.edu>
@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: hera
Compiler: gnu
Job: BL
[BL] Repo location: /scratch1/BMC/gmtb/RT/auto_RT/Pull_Requests/1298047098/20230410171512/ufs-weather-model
[BL] Baseline creation and move successful
[RT] Repo location: /scratch1/BMC/gmtb/RT/auto_RT/Pull_Requests/1298047098/20230410184507/ufs-weather-model
Please make changes and add the following label back: hera-gnu-BL

on-behalf-of @NCAR <dswales@ucar.edu>
@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: hera
Compiler: intel
Job: BL
[BL] Repo location: /scratch1/BMC/gmtb/RT/auto_RT/Pull_Requests/1298047098/20230410194351/ufs-weather-model
[BL] Baseline creation and move successful
[RT] Repo location: /scratch1/BMC/gmtb/RT/auto_RT/Pull_Requests/1298047098/20230410204832/ufs-weather-model
Please make changes and add the following label back: hera-intel-BL

on-behalf-of @NCAR <dswales@ucar.edu>
@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: gnu
Job: BL
[BL] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230411111508/ufs-weather-model
[BL] Baseline creation and move successful
[RT] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230411120414/ufs-weather-model
Please make changes and add the following label back: cheyenne-gnu-BL

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: intel
Job: BL
Please make changes and add the following label back: cheyenne-intel-BL

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: intel
Job: BL
[BL] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230411124510/ufs-weather-model
[BL] Baseline creation and move successful
[RT] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230411140126/ufs-weather-model
[RT] Error: Test control_stochy 027 failed in check_result failed
[RT] Error: Test control_stochy 027 failed in run_test failed
[RT] Error: Test regional_control 039 failed in check_result failed
[RT] Error: Test regional_control 039 failed in run_test failed
[RT] Error: Test regional_decomp 041 failed in check_result failed
[RT] Error: Test regional_decomp 041 failed in run_test failed
[RT] Error: Test regional_2threads 042 failed in check_result failed
[RT] Error: Test regional_2threads 042 failed in run_test failed
[RT] Error: Test regional_2dwrtdecomp 045 failed in check_result failed
[RT] Error: Test regional_2dwrtdecomp 045 failed in run_test failed
Please make changes and add the following label back: cheyenne-intel-BL

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: intel
Job: BL
[BL] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230411154507/ufs-weather-model
[BL] Error: Test compile_008 failed in run_compile failed
Please make changes and add the following label back: cheyenne-intel-BL

@dustinswales
Copy link
Collaborator

Automated RT Failure Notification
Machine: cheyenne
Compiler: intel
Job: BL
[BL] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230412083010/ufs-weather-model
[BL] Baseline creation and move successful
[RT] Repo location: /glade/scratch/epicufsrt/GMTB/ufs-weather-model/RT/auto_RT/Pull_Requests/1298047098/20230412094450/ufs-weather-model
[RT] Error: Test hafs_regional_atm_wav 114 failed in check_result failed
[RT] Error: Test hafs_regional_atm_wav 114 failed in run_test failed
Please make changes and add the following label back: cheyenne-intel-BL

@grantfirl
Copy link
Collaborator Author

@dustinswales The hafs_regional_atm_wav run failed on the verification step of the baseline creation, but I reran the test individually, and it passed. Do you think it is OK to merge without including a new successful Cheyenne/Intel log? I'd really not like to run another round of RTs, especially since things seem to fail randomly.

@dustinswales
Copy link
Collaborator

@grantfirl I think it's fine. To be honest, I don't see an advantage of keeping the logs under version control, at least with the troubles we have on Cheyenne.

@dustinswales dustinswales self-requested a review April 12, 2023 19:32
@grantfirl
Copy link
Collaborator Author

@dustinswales I'm going to keep the diff in tests/fv3_conf/compile_qsub.IN_cheyenne while we work through these PRs to save some work.

@grantfirl grantfirl merged commit 8b276be into NCAR:main Apr 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants