Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/single precision (ESMF init errors) #2

Conversation

climbfuji
Copy link

WORK IN PROGRESS

Note. The current code in @SamuelTrahanNOAA's branches (recursive clone of branch sing_prec_from_main of the ufs-weather-model, see PR ufs-community/ufs-weather-model#1215) crashes in the model initialization step, but not in the physics - it crashes deep inside the ESMF grid/field initialization (cont'd below):

20220522 105805.131 INFO             PET134 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20220522 105805.131 INFO             PET134 !!! THE ESMF_LOG IS SET TO OUTPUT ALL LOG MESSAGES !!!
20220522 105805.131 INFO             PET134 !!!     THIS MAY CAUSE SLOWDOWN IN PERFORMANCE     !!!
20220522 105805.131 INFO             PET134 !!! FOR PRODUCTION RUNS, USE:                      !!!
20220522 105805.131 INFO             PET134 !!!                   ESMF_LOGKIND_Multi_On_Error  !!!
20220522 105805.131 INFO             PET134 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20220522 105805.131 INFO             PET134 Running with ESMF Version   : v8.3.0b09
20220522 105805.131 INFO             PET134 ESMF library build date/time: "Mar 29 2022" "17:20:36"
20220522 105805.131 INFO             PET134 ESMF library build location : /glade/work/jongkim/stacks/hash/hpc-stack-esmf/pkg/v8.3.0b09
20220522 105805.131 INFO             PET134 ESMF_COMM                   : mpt
20220522 105805.131 INFO             PET134 ESMF_MOAB                   : enabled
20220522 105805.131 INFO             PET134 ESMF_LAPACK                 : enabled
20220522 105805.131 INFO             PET134 ESMF_NETCDF                 : enabled
20220522 105805.131 INFO             PET134 ESMF_PNETCDF                : disabled
20220522 105805.131 INFO             PET134 ESMF_PIO                    : enabled
20220522 105805.131 INFO             PET134 ESMF_YAMLCPP                : enabled
20220522 105805.132 TRACE            PET134  ESMF_LogSet() !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20220522 105805.132 TRACE            PET134  ESMF_LogSet() !!!       TRACING is disabled         !!!
20220522 105805.132 TRACE            PET134  ESMF_LogSet() !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
20220522 105805.394 INFO             PET134 ReadAttributes DRIVER_attributes:: start:
20220522 105805.394 INFO             PET134 ReadAttributes DRIVER_attributes:: end:
20220522 105805.394 INFO             PET134 ReadAttributes ALLCOMP_attributes:: start:
20220522 105805.394 INFO             PET134 ReadAttributes ALLCOMP_attributes:: end:
20220522 105805.394 INFO             PET134 (module_EARTH_GRID_COMP.F90:InitRestart): called
20220522 105805.394 INFO             PET134 No start_type attribute found, setting read_restart false
20220522 105805.394 INFO             PET134 Set attribute read_restart in driver
20220522 105805.400 INFO             PET134 (module_EARTH_GRID_COMP.F90:AddAttributes): called
20220522 105805.400 INFO             PET134 Added attribute read_restart to ATM
20220522 105805.400 INFO             PET134 ReadAttributes ATM_attributes:: start:
20220522 105805.400 INFO             PET134 ReadAttributes ATM_attributes:: end:
20220522 105805.400 INFO             PET134 ReadAttributes ATM_modelio:: start:
20220522 105805.400 INFO             PET134 ReadAttributes ATM_modelio:: end:
20220522 105805.400 INFO             PET134 ReadAttributes ALLCOMP_attributes:: start:
20220522 105805.400 INFO             PET134 ReadAttributes ALLCOMP_attributes:: end:
20220522 105805.424 INFO             PET134 (fv3_cap:InitializeAdvertise) cplprint_flag =      F
20220522 105805.424 INFO             PET134 (fv3_cap:InitializeAdvertise) dbug =      0
20220522 105805.427 DEBUG            PET134 Generic SetVM() is executing for: fv3_fcst
20220522 105805.427 DEBUG            PET134 Generic SetVM() found no NUOPC Hint for: fv3_fcst
20220522 105805.427 DEBUG            PET134 Generic SetVM() therefore is NOT calling ESMF_GridCompSetVMMaxPEs() for: fv3_fcst
20220522 105836.613 ERROR            PET134 ESMCI_Grid.C:3685 ESMCI::Grid::setCoordArray() Arguments are incompatible  - - Array and Grid TypeKind mismatch
20220522 105836.614 ERROR            PET134 ESMCI_Grid_F.C:1091 c_esmc_gridsetfromarray() Arguments are incompatible  - Internal subroutine call returned Error
20220522 105836.614 ERROR            PET134 ESMF_Grid.F90:25269 ESMF_GridSetCoordFromArray Arguments are incompatible  - Internal subroutine call returned Error
20220522 105836.617 ERROR            PET134 /glade/work/heinzell/fv3/ufs-weather-model/ufs-weather-model-rap-control-32bit/gnu/FV3/module_fcst_grid_comp.F90:233 Arguments are incompatible  - Passing error in return code
20220522 105836.617 ERROR            PET134 /glade/work/heinzell/fv3/ufs-weather-model/ufs-weather-model-rap-control-32bit/gnu/FV3/module_fcst_grid_comp.F90:837 Arguments are incompatible  - Passing error in return code
20220522 105836.617 ERROR            PET134 /glade/work/heinzell/fv3/ufs-weather-model/ufs-weather-model-rap-control-32bit/gnu/FV3/fv3_cap.F90:377 Arguments are incompatible  - Passing error in return code
20220522 105836.617 ERROR            PET134 ATM:src/addon/NUOPC/src/NUOPC_ModelBase.F90:700 Arguments are incompatible  - Passing error in return code
20220522 105836.617 ERROR            PET134 EARTH Grid Comp:src/addon/NUOPC/src/NUOPC_Driver.F90:2577 Arguments are incompatible  - Phase 'IPDvXp01' Initialize for modelComp 1: ATM did not return ESMF_SUCCESS
20220522 105836.617 ERROR            PET134 EARTH Grid Comp:src/addon/NUOPC/src/NUOPC_Driver.F90:1286 Arguments are incompatible  - Passing error in return code
20220522 105836.617 ERROR            PET134 EARTH Grid Comp:src/addon/NUOPC/src/NUOPC_Driver.F90:457 Arguments are incompatible  - Passing error in return code
20220522 105836.617 ERROR            PET134 /glade/work/heinzell/fv3/ufs-weather-model/ufs-weather-model-rap-control-32bit/gnu/driver/UFS.F90:386 Arguments are incompatible  - Aborting UFS
20220522 105836.617 INFO             PET134 Finalizing ESMF

This PR is a fool's attempt from someone with no idea of ESMF and how the coupling is realized to make this work. But it doesn't, still the same error. I actually believe that it would be better to improve the interface between the physics fields and the coupling fields, so that:

  • Physics use their own variables in the precision requested for physics
  • Coupling fields are what the coupling framework in the host model defines (double precision in the UFS?)
  • Physics fields are updated from coupling fields and vice versa outside of the physics. Right now, coupling fields are passed to the physics, and the physics are required to "know" which arrays to populate based on a few logical flags. This seems wrong to me, there should be a separation of concerns?

@climbfuji
Copy link
Author

This PR is no longer needed, these errors have been fixed (much better than what I had suggested).

@climbfuji climbfuji closed this Jun 3, 2022
@climbfuji climbfuji deleted the feature/single_precision branch June 3, 2022 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant