-
Notifications
You must be signed in to change notification settings - Fork 235
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix crash issue in MOM_SIS when compiling with openmp #6
Conversation
I have tested this branch. It resolves the crash issues for ocean_ice examples and preserves the timestats when compiled with -openmp. Since this branch fixes the problems with ocean_ice experiments I am going to push it to dev/master |
fix crash issue in MOM_SIS when compiling with openmp
Draft of frepp files for CM2G63L They worked for CM4i
- NOAA-GFDL/MOM6@ae22a0f Merge pull request mom-ocean#578 from NCAR/dev/ncar - NOAA-GFDL/MOM6@f4f882f Merge pull request mom-ocean#23 from gustavo-marques/combine_mct_modules - NOAA-GFDL/MOM6@a85b728 Merge MCT modules - NOAA-GFDL/MOM6@ba5b5ba Merge pull request mom-ocean#22 from gustavo-marques/check_fluxes - NOAA-GFDL/MOM6@f5a8483 Adds option to write restart files - NOAA-GFDL/MOM6@8745b4b Fills additional IOB fluxes - NOAA-GFDL/MOM6@f6b22ff Merge branch 'dev/ncar' into check_fluxes - NOAA-GFDL/MOM6@369d5d6 Merge pull request mom-ocean#21 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@2582e0a Adds run paramaters to control SW decomposition - NOAA-GFDL/MOM6@58f78b9 Fill IOB, some fluxes still need to be checked - NOAA-GFDL/MOM6@06a30ab Fixed clock and timing - NOAA-GFDL/MOM6@1bca79b Merge pull request mom-ocean#20 from gustavo-marques/fix_merge - NOAA-GFDL/MOM6@fb89b08 Adds code that got lost during merge - NOAA-GFDL/MOM6@7480492 Merge pull request mom-ocean#19 from gustavo-marques/new_merge - NOAA-GFDL/MOM6@be1e6cf Merge branch 'dev/master' into dev/ncar - NOAA-GFDL/MOM6@76565cb Merge pull request mom-ocean#17 from gustavo-marques/fix_ssh_slopes - NOAA-GFDL/MOM6@c04f081 Fix ssh slope calculation - NOAA-GFDL/MOM6@17d4586 Merge pull request mom-ocean#16 from adcroft/ncar-cherry-picked-fixes - NOAA-GFDL/MOM6@be46fee (*)Fixes gprime(1) when no layer coordinates are set - NOAA-GFDL/MOM6@fccb7ae Avoids a SEGV when OBC are not in use with bi-harmonic friction - NOAA-GFDL/MOM6@f42c970 Merge pull request mom-ocean#15 from gustavo-marques/small_fixes - NOAA-GFDL/MOM6@31787c2 Adds reference to set_time that was lost during a conflict handling - NOAA-GFDL/MOM6@d77936e Fix intent(inout) in ice_ocean_boundary - NOAA-GFDL/MOM6@b403edd Merge pull request mom-ocean#14 from gustavo-marques/docs - NOAA-GFDL/MOM6@4d346ca Merge branch 'dev/ncar' into docs - NOAA-GFDL/MOM6@f7c26a5 Clean the module and start doxygenizing it - NOAA-GFDL/MOM6@cd58e2a Merge pull request mom-ocean#13 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@37a53bc reorganize file structure - NOAA-GFDL/MOM6@e6c5ddd Merge pull request mom-ocean#12 from gustavo-marques/fill_ice_ocean_bnd - NOAA-GFDL/MOM6@6b6289a Remove trailing whitespace - NOAA-GFDL/MOM6@7ce40e5 Fix undefined reference to fill_ice_ocean_bnd - NOAA-GFDL/MOM6@d517719 Merge branch 'dev/ncar' into fill_ice_ocean_bnd - NOAA-GFDL/MOM6@b69ab33 Filling ice_ocean_bnd - Draft - NOAA-GFDL/MOM6@a306c90 Merge pull request mom-ocean#11 from adcroft/use-ocn-public-for-surface - NOAA-GFDL/MOM6@c83e238 Correct indexing when using ocean_public_type - NOAA-GFDL/MOM6@1f39c84 Replaces use of "surface" with "ocean_public_type" - NOAA-GFDL/MOM6@5cd8d7a Filling ice_ocean_bnd, still needs work - NOAA-GFDL/MOM6@3ceef26 Merge pull request mom-ocean#10 from adcroft/sketch-out-mct-run - NOAA-GFDL/MOM6@714e8d6 Adds placeholders for stepping foward MOM - NOAA-GFDL/MOM6@7f2207c Merge pull request #7 from adcroft/fix-mct-whitespace - NOAA-GFDL/MOM6@a1db77f Cleans up white space - NOAA-GFDL/MOM6@5700d67 Merge pull request #6 from adcroft/add-use-only-for-imports - NOAA-GFDL/MOM6@beac3ed Adds "only" for all use statements - NOAA-GFDL/MOM6@231d19e Merge pull request #5 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@3a923a2 Got initialization working - NOAA-GFDL/MOM6@8b38141 Wrapped global data - NOAA-GFDL/MOM6@970f848 added debugging - NOAA-GFDL/MOM6@2540ed8 Filled empty subroutines in ocn_comp_mct.F90 - NOAA-GFDL/MOM6@ed3ed30 Added ocn_import_export module - NOAA-GFDL/MOM6@aba7dc2 Completed ocn_init_mct - NOAA-GFDL/MOM6@5df3bc3 Added method to point ponters to members of ocean_state_type - NOAA-GFDL/MOM6@57e6086 Added methods to return domain/grid shape - NOAA-GFDL/MOM6@37e7356 Added ocn_import_export.F90 - NOAA-GFDL/MOM6@2b794da Merge pull request #4 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@fbd405b Added coupler indices module - NOAA-GFDL/MOM6@fa86163 Merge pull request #3 from NCAR/dev/gfdl - NOAA-GFDL/MOM6@e8f2999 Merge pull request #2 from gustavo-marques/temp_salt_z_init_file - NOAA-GFDL/MOM6@86936ba Adds option to read initial temp and salt from separate files - NOAA-GFDL/MOM6@54f7976 Most initialization working - NOAA-GFDL/MOM6@426357c Time and calendar initialization in ocn_comp_mct.F90 - NOAA-GFDL/MOM6@346097d Moved blank ocn_comp_mct.F90 - NOAA-GFDL/MOM6@405fecb Merge pull request #1 from gustavo-marques/ice_shelf - NOAA-GFDL/MOM6@711a613 Removes frazil's contribution to evaporation
- Updates MOM_parameter_doc files. - NOAA-GFDL/MOM6@cde6d3f Merge branch 'dev/master' into dev/gfdl - NOAA-GFDL/MOM6@ae22a0f Merge pull request mom-ocean#578 from NCAR/dev/ncar - NOAA-GFDL/MOM6@f4f882f Merge pull request mom-ocean#23 from gustavo-marques/combine_mct_modules - NOAA-GFDL/MOM6@a85b728 Merge MCT modules - NOAA-GFDL/MOM6@ba5b5ba Merge pull request mom-ocean#22 from gustavo-marques/check_fluxes - NOAA-GFDL/MOM6@f5a8483 Adds option to write restart files - NOAA-GFDL/MOM6@8745b4b Fills additional IOB fluxes - NOAA-GFDL/MOM6@f6b22ff Merge branch 'dev/ncar' into check_fluxes - NOAA-GFDL/MOM6@369d5d6 Merge pull request mom-ocean#21 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@2582e0a Adds run paramaters to control SW decomposition - NOAA-GFDL/MOM6@58f78b9 Fill IOB, some fluxes still need to be checked - NOAA-GFDL/MOM6@06a30ab Fixed clock and timing - NOAA-GFDL/MOM6@1bca79b Merge pull request mom-ocean#20 from gustavo-marques/fix_merge - NOAA-GFDL/MOM6@fb89b08 Adds code that got lost during merge - NOAA-GFDL/MOM6@7480492 Merge pull request mom-ocean#19 from gustavo-marques/new_merge - NOAA-GFDL/MOM6@be1e6cf Merge branch 'dev/master' into dev/ncar - NOAA-GFDL/MOM6@76565cb Merge pull request mom-ocean#17 from gustavo-marques/fix_ssh_slopes - NOAA-GFDL/MOM6@c04f081 Fix ssh slope calculation - NOAA-GFDL/MOM6@17d4586 Merge pull request mom-ocean#16 from adcroft/ncar-cherry-picked-fixes - NOAA-GFDL/MOM6@be46fee (*)Fixes gprime(1) when no layer coordinates are set - NOAA-GFDL/MOM6@fccb7ae Avoids a SEGV when OBC are not in use with bi-harmonic friction - NOAA-GFDL/MOM6@f42c970 Merge pull request mom-ocean#15 from gustavo-marques/small_fixes - NOAA-GFDL/MOM6@31787c2 Adds reference to set_time that was lost during a conflict handling - NOAA-GFDL/MOM6@d77936e Fix intent(inout) in ice_ocean_boundary - NOAA-GFDL/MOM6@b403edd Merge pull request mom-ocean#14 from gustavo-marques/docs - NOAA-GFDL/MOM6@4d346ca Merge branch 'dev/ncar' into docs - NOAA-GFDL/MOM6@f7c26a5 Clean the module and start doxygenizing it - NOAA-GFDL/MOM6@cd58e2a Merge pull request mom-ocean#13 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@37a53bc reorganize file structure - NOAA-GFDL/MOM6@e6c5ddd Merge pull request mom-ocean#12 from gustavo-marques/fill_ice_ocean_bnd - NOAA-GFDL/MOM6@6b6289a Remove trailing whitespace - NOAA-GFDL/MOM6@7ce40e5 Fix undefined reference to fill_ice_ocean_bnd - NOAA-GFDL/MOM6@d517719 Merge branch 'dev/ncar' into fill_ice_ocean_bnd - NOAA-GFDL/MOM6@b69ab33 Filling ice_ocean_bnd - Draft - NOAA-GFDL/MOM6@a306c90 Merge pull request mom-ocean#11 from adcroft/use-ocn-public-for-surface - NOAA-GFDL/MOM6@c83e238 Correct indexing when using ocean_public_type - NOAA-GFDL/MOM6@1f39c84 Replaces use of "surface" with "ocean_public_type" - NOAA-GFDL/MOM6@5cd8d7a Filling ice_ocean_bnd, still needs work - NOAA-GFDL/MOM6@3ceef26 Merge pull request mom-ocean#10 from adcroft/sketch-out-mct-run - NOAA-GFDL/MOM6@714e8d6 Adds placeholders for stepping foward MOM - NOAA-GFDL/MOM6@7f2207c Merge pull request #7 from adcroft/fix-mct-whitespace - NOAA-GFDL/MOM6@a1db77f Cleans up white space - NOAA-GFDL/MOM6@5700d67 Merge pull request #6 from adcroft/add-use-only-for-imports - NOAA-GFDL/MOM6@beac3ed Adds "only" for all use statements - NOAA-GFDL/MOM6@231d19e Merge pull request #5 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@3a923a2 Got initialization working - NOAA-GFDL/MOM6@8b38141 Wrapped global data - NOAA-GFDL/MOM6@970f848 added debugging - NOAA-GFDL/MOM6@2540ed8 Filled empty subroutines in ocn_comp_mct.F90 - NOAA-GFDL/MOM6@ed3ed30 Added ocn_import_export module - NOAA-GFDL/MOM6@aba7dc2 Completed ocn_init_mct - NOAA-GFDL/MOM6@5df3bc3 Added method to point ponters to members of ocean_state_type - NOAA-GFDL/MOM6@57e6086 Added methods to return domain/grid shape - NOAA-GFDL/MOM6@37e7356 Added ocn_import_export.F90 - NOAA-GFDL/MOM6@2b794da Merge pull request #4 from alperaltuntas/dev/ncar - NOAA-GFDL/MOM6@fbd405b Added coupler indices module - NOAA-GFDL/MOM6@fa86163 Merge pull request #3 from NCAR/dev/gfdl - NOAA-GFDL/MOM6@e8f2999 Merge pull request #2 from gustavo-marques/temp_salt_z_init_file - NOAA-GFDL/MOM6@86936ba Adds option to read initial temp and salt from separate files - NOAA-GFDL/MOM6@54f7976 Most initialization working - NOAA-GFDL/MOM6@426357c Time and calendar initialization in ocn_comp_mct.F90 - NOAA-GFDL/MOM6@346097d Moved blank ocn_comp_mct.F90 - NOAA-GFDL/MOM6@405fecb Merge pull request #1 from gustavo-marques/ice_shelf - NOAA-GFDL/MOM6@711a613 Removes frazil's contribution to evaporation
Add 'mfo' (mass transport through straits' to refine_diag
Create ncar fork of doxygen
Change default for USE_NET_FW_ADJUSTMENT_SIGN_BUG in drivers
…-2021-02-19 Dev gfdl main candidate 2021 02 19
Add initialization tests using CS%initialized
Using tikal_201304 and MOM6 dev/master, all MOM6_SIS test cases crash when compiled with -openmp with:
FATAL from PE 30: NaN in input field of reproducing_sum(_2d).
This is the xml and fre commands to reproduce the issue:
/ncrc/home2/Niki.Zadeh/frerts/tikal_201403_devmaster20140313/_0/MOM6_SIS.xml.20140313201957
fremake --execute --nolink -x /ncrc/home2/Niki.Zadeh/frerts/tikal_201403_devmaster20140313/_0/MOM6_SIS.xml.20140313201957 -p ncrc2.intel -t repro-openmp MOM6_SIS_libs_compile
fremake --execute -x /ncrc/home2/Niki.Zadeh/frerts/tikal_201403_devmaster20140313/_0/MOM6_SIS.xml.20140313201957 -p ncrc2.intel -t repro-openmp MOM6_SIS_compile
frerun -s -r basic -u --no-transfer --partition t1 -x /ncrc/home2/Niki.Zadeh/frerts/tikal_201403_devmaster20140313/_0/MOM6_SIS.xml.20140313201957 -p ncrc2.intel -t repro-openmp MOM6_GOLD_SIS
Or you can try the following command to build and run the test:
/ncrc/home2/Niki.Zadeh/fre/fre-commands/bin/frerts_batch.csh -p ncrc2.intel -t repro-openmp --release tikal_201403 --fre_stem tikal_201403_devmaster20140313 --mom_git_tag 'dev\/master' -x /ncrc/home2/Niki.Zadeh/xmls/tikal_f1/xml/MOM6_SIS.xml --frerts_ops '--no_rts,--compile,-l,MOM6_SIS_libs_compile,--fre_ops,-u;--no-transfer;--partition=t1,--do_frecheck,--reference_tag,tikal_mom6_2014.01.17' --fre_version 'fre\/bronx-7' --mom_git_tag 'dev\/master' --debuglevel _0 --interactive
MOM6_GOLD_SIS
These the stdouts for openmp and regular runs:
/lustre/f1/Niki.Zadeh/tikal_201403_devmaster20140313_0/MOM6_GOLD_SIS/ncrc2.intel-repro-openmp/stdout/run/MOM6_GOLD_SIS_1x0m20d_32pe.o5031341
/lustre/f1/Niki.Zadeh/tikal_201403_devmaster20140313_0/MOM6_GOLD_SIS/ncrc2.intel-repro/stdout/run/MOM6_GOLD_SIS_1x0m20d_32pe.o5031270