Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Getting Segmentation Fault for ocean_only/benchmark #229

Closed
vineetmoraybfab opened this issue May 9, 2018 · 3 comments
Closed

Getting Segmentation Fault for ocean_only/benchmark #229

vineetmoraybfab opened this issue May 9, 2018 · 3 comments

Comments

@vineetmoraybfab
Copy link

vineetmoraybfab commented May 9, 2018 •
edited
Loading

Running ocean_only/benchmark on Xeon Broadwell processors is printing segmentation fault.
Have built the application using intel compilers and optimization flags: "-O3 -xHost(which comes out to be -msse2"

NOTE from PE 0: MPP_DOMAINS_SET_STACK_SIZE: stack size set to 32768.
&MPP_IO_NML
HEADER_BUFFER_VAL = 16384,
GLOBAL_FIELD_ON_ROOT_PE = T,
IO_CLOCKS_ON = F,
SHUFFLE = 0,
DEFLATE_LEVEL = -1,
CF_COMPLIANCE = F
/
NOTE from PE 0: MPP_IO_SET_STACK_SIZE: stack size set to 131072.
NOTE from PE 0: MPP_DOMAINS_SET_STACK_SIZE: stack size set to 955296.
NOTE from PE 0: ======== Model being driven by MOM_driver ========
NOTE from PE 0: callTree: o Program MOM_main, MOM_driver.F90
NOTE from PE 0: open_param_file: MOM_input has been opened successfully.
NOTE from PE 0: open_param_file: MOM_override has been opened successfully.
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
NOTE from PE 0: diag_manager_mod::diag_manager_init: prepend_date only supported when diag_manager_init is called with time_init present.
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
MOM Date 1/01/01 00:00:00 0: En 5.483130E-01, MaxCFL 0.00000, Mass 7.909733697458E+19, Salt 35.00000000000, Temp 5.06383031873
Total Energy: 4402CF0C41B7B49B 4.3370095656201052E+19
Total Mass: 7.9097336974577992E+19, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
Total Salt: 2.7684067941102295E+18, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
Total Heat: 1.5721018104275223E+24, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
Total age: 0.0000000000000000E+00 yr kg
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FA70D8155E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom_ 672 MOM.F90
MOM6 00000000007C1EE0 MAIN__ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FA70D162C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F77097315E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F770907EC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F5A039065E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F5A03253C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FB384FA35E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FB3848F0C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FA0782D35E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FA077C20C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F3651D335E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F3651680C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F4F74FD55E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F4F74922C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FB0E27A85E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FB0E20F5C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FEAEDFA95E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FEAED8F6C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F6BF91A65E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F6BF8AF3C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FBBB54485E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FBBB4D95C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FA1C02475E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FA1BFB94C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F1C8499A5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F1C842E7C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FAB7B28C5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FAB7ABD9C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F2C8C5795E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F2C8BEC6C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F55E1D5B5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F55E16A8C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FB07F10D5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FB07EA5AC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F5CBC36A5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F5CBBCB7C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F3E865C55E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F3E85F12C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F95C77645E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F95C70B1C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FF2C10565E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FF2C09A3C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F79723765E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F7971CC3C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F0DA48E65E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F0DA4233C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F1FED0355E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F1FEC982C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FE3D2D5D5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FE3D26AAC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F374BFD75E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F374B924C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FB4C50035E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FB4C4950C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F727FA875E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F727F3D4C05 __libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown

This is with DEBUG=True in MOM_override.

NOTE from PE 0: MPP_DOMAINS_SET_STACK_SIZE: stack size set to 32768.
&MPP_IO_NML
HEADER_BUFFER_VAL = 16384,
GLOBAL_FIELD_ON_ROOT_PE = T,
IO_CLOCKS_ON = F,
SHUFFLE = 0,
DEFLATE_LEVEL = -1,
CF_COMPLIANCE = F
/
NOTE from PE 0: MPP_IO_SET_STACK_SIZE: stack size set to 131072.
NOTE from PE 0: MPP_DOMAINS_SET_STACK_SIZE: stack size set to 955296.
NOTE from PE 0: ======== Model being driven by MOM_driver ========
NOTE from PE 0: callTree: o Program MOM_main, MOM_driver.F90
NOTE from PE 0: open_param_file: MOM_input has been opened successfully.
NOTE from PE 0: open_param_file: MOM_override has been opened successfully.
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
NOTE from PE 0: diag_manager_mod::diag_manager_init: prepend_date only supported when diag_manager_init is called with time_init present.
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
h-point: mean= 2.5343143909337967E+04 min= 2.1039476278425322E+04 max= 2.7829208694415916E+04 x MOM_grid_init/set_grid_metrics: d[xy]T
h-point: c= 1940400 sw= 1939680 se= 1939680 nw= 1942560 ne= 1942560 x MOM_grid_init/set_grid_metrics: d[xy]T
h-point: mean= 2.5343143909337967E+04 min= 2.1039476278425318E+04 max= 2.7829208694415920E+04 y MOM_grid_init/set_grid_metrics: d[xy]T
h-point: c= 1935720 sw= 1933920 se= 1933920 nw= 1938600 ne= 1938600 y MOM_grid_init/set_grid_metrics: d[xy]T
u-point: mean= 2.5343143909337967E+04 min= 2.1039476278425322E+04 max= 2.7829208694415916E+04 u MOM_grid_init/set_grid_metrics: dxC[uv]
u-point: c= 1940400 sw= 1939680 se= 1939680 nw= 1942560 ne= 1942560 u MOM_grid_init/set_grid_metrics: dxC[uv]
v-point: mean= 2.5362046189593148E+04 min= 2.1009424704132529E+04 max= 2.7829274923049583E+04 v MOM_grid_init/set_grid_metrics: dxC[uv]
v-point: c= 1938960 sw= 1939320 se= 1939320 nw= 1938960 ne= 1938960 v MOM_grid_init/set_grid_metrics: dxC[uv]
u-point: mean= 2.5343143909337967E+04 min= 2.1039476278425318E+04 max= 2.7829208694415920E+04 u MOM_grid_init/set_grid_metrics: dxC[uv]
u-point: c= 1935720 sw= 1933920 se= 1933920 nw= 1938600 ne= 1938600 u MOM_grid_init/set_grid_metrics: dxC[uv]
v-point: mean= 2.5362046189593148E+04 min= 2.1009424704132529E+04 max= 2.7829274923049583E+04 v MOM_grid_init/set_grid_metrics: dxC[uv]
v-point: c= 1937160 sw= 1937520 se= 1937520 nw= 1937160 ne= 1937160 v MOM_grid_init/set_grid_metrics: dxC[uv]
B-point: mean= 2.5362046189593148E+04 min= 2.1009424704132529E+04 max= 2.7829274923049583E+04 x MOM_grid_init/set_grid_metrics: dxB[uv]
B-point: c= 1937160 sw= 1937520 se= 1937520 nw= 1937160 ne= 1937160 x MOM_grid_init/set_grid_metrics: dxB[uv]
B-point: mean= 2.5362046189593148E+04 min= 2.1009424704132529E+04 max= 2.7829274923049583E+04 y MOM_grid_init/set_grid_metrics: dxB[uv]
B-point: c= 1938960 sw= 1939320 se= 1939320 nw= 1938960 ne= 1938960 y MOM_grid_init/set_grid_metrics: dxB[uv]
h-point: mean= 3.9743232448686427E-05 min= 3.5933468715574921E-05 max= 4.7529700205771659E-05 x MOM_grid_init/set_grid_metrics: Id[xy]T
h-point: c= 2051280 sw= 2052360 se= 2052360 nw= 2053080 ne= 2053080 x MOM_grid_init/set_grid_metrics: Id[xy]T
h-point: mean= 3.9743232448686427E-05 min= 3.5933468715574921E-05 max= 4.7529700205771672E-05 y MOM_grid_init/set_grid_metrics: Id[xy]T
h-point: c= 2056680 sw= 2058480 se= 2058480 nw= 2058120 ne= 2058120 y MOM_grid_init/set_grid_metrics: Id[xy]T
u-point: mean= 3.9743232448686427E-05 min= 3.5933468715574921E-05 max= 4.7529700205771659E-05 u MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
u-point: c= 2051280 sw= 2052360 se= 2052360 nw= 2053080 ne= 2053080 u MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
v-point: mean= 3.9710926189510507E-05 min= 3.5933383200427922E-05 max= 4.7597685994862164E-05 v MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
v-point: c= 2051640 sw= 2049840 se= 2049840 nw= 2050200 ne= 2050200 v MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
u-point: mean= 3.9743232448686427E-05 min= 3.5933468715574921E-05 max= 4.7529700205771672E-05 u MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
u-point: c= 2056680 sw= 2058480 se= 2058480 nw= 2058120 ne= 2058120 u MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
v-point: mean= 3.9710926189510507E-05 min= 3.5933383200427922E-05 max= 4.7597685994862164E-05 v MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
v-point: c= 2056320 sw= 2054520 se= 2054520 nw= 2054880 ne= 2054880 v MOM_grid_init/set_grid_metrics: Id[xy]C[uv]
B-point: mean= 3.9710926189510507E-05 min= 3.5933383200427922E-05 max= 4.7597685994862164E-05 x MOM_grid_init/set_grid_metrics: Id[xy]B[uv]
B-point: c= 2056320 sw= 2054520 se= 2054520 nw= 2054880 ne= 2054880 x MOM_grid_init/set_grid_metrics: Id[xy]B[uv]
B-point: mean= 3.9710926189510507E-05 min= 3.5933383200427922E-05 max= 4.7597685994862164E-05 y MOM_grid_init/set_grid_metrics: Id[xy]B[uv]
B-point: c= 2051640 sw= 2049840 se= 2049840 nw= 2050200 ne= 2050200 y MOM_grid_init/set_grid_metrics: Id[xy]B[uv]
h-point: mean= 6.4666873870187151E+08 min= 4.4265956207042176E+08 max= 7.7446485655735457E+08 MOM_grid_init/set_grid_metrics: areaT
h-point: c= 1949400 sw= 1948680 se= 1948680 nw= 1950480 ne= 1950480 MOM_grid_init/set_grid_metrics: areaT
B-point: mean= 6.4759218620522928E+08 min= 4.4139592639861417E+08 max= 7.7446854274267638E+08 MOM_grid_init/set_grid_metrics: areaBu
B-point: c= 1931400 sw= 1931400 se= 1931400 nw= 1927800 ne= 1927800 MOM_grid_init/set_grid_metrics: areaBu
h-point: mean= 1.5914447124361934E-09 min= 1.2912141739332015E-09 max= 2.2590724016505310E-09 MOM_grid_init/set_grid_metrics: IareaT
h-point: c= 2076480 sw= 2076120 se= 2076120 nw= 2077560 ne= 2077560 MOM_grid_init/set_grid_metrics: IareaT
B-point: mean= 1.5887472392869648E-09 min= 1.2912080282287958E-09 max= 2.2655397120654979E-09 MOM_grid_init/set_grid_metrics: IareaBu
B-point: c= 2087640 sw= 2085480 se= 2085480 nw= 2089080 ne= 2089080 MOM_grid_init/set_grid_metrics: IareaBu
h-point: mean= 4.5000000000000000E+01 min= 1.2500000000000000E-01 max= 8.9874999999999986E+01 MOM_grid_init/set_grid_metrics: geoLonT
h-point: c= 800460 sw= 792720 se= 800280 nw= 792720 ne= 800280 MOM_grid_init/set_grid_metrics: geoLonT
h-point: mean= -2.1448475528681353E+01 min= -4.0885463109809884E+01 max= -1.2499990084080956E-01 MOM_grid_init/set_grid_metrics: geoLatT
h-point: c= 1850040 sw= 1843560 se= 1843560 nw= 1855080 ne= 1855080 MOM_grid_init/set_grid_metrics: geoLatT
B-point: mean= 4.5125000000000000E+01 min= 0.0000000000000000E+00 max= 9.0000000000000000E+01 MOM_grid_init/set_grid_metrics: geoLonBu
B-point: c= 721260 sw= 720180 se= 728640 nw= 720180 ne= 728640 MOM_grid_init/set_grid_metrics: geoLonBu
B-point: mean= -2.1334599933236913E+01 min= -4.0979898069620141E+01 max= 0.0000000000000000E+00 MOM_grid_init/set_grid_metrics: geoLatBu
B-point: c= 1850760 sw= 1861560 se= 1861560 nw= 1855800 ne= 1855800 MOM_grid_init/set_grid_metrics: geoLatBu
u-point: mean= 4.5125000000000000E+01 min= 0.0000000000000000E+00 max= 9.0000000000000000E+01 u MOM_grid_init/set_grid_metrics: geoLonC[uv]
u-point: c= 721260 sw= 720180 se= 728640 nw= 720180 ne= 728640 u MOM_grid_init/set_grid_metrics: geoLonC[uv]
v-point: mean= 4.5000000000000000E+01 min= 1.2500000000000000E-01 max= 8.9874999999999986E+01 v MOM_grid_init/set_grid_metrics: geoLonC[uv]
v-point: c= 800460 sw= 792720 se= 800280 nw= 792720 ne= 800280 v MOM_grid_init/set_grid_metrics: geoLonC[uv]
u-point: mean= -2.1448475528681353E+01 min= -4.0885463109809884E+01 max= -1.2499990084080956E-01 u MOM_grid_init/set_grid_metrics: geoLatC[uv]
u-point: c= 1850040 sw= 1843560 se= 1843560 nw= 1855080 ne= 1855080 u MOM_grid_init/set_grid_metrics: geoLatC[uv]
v-point: mean= -2.1334599933236913E+01 min= -4.0979898069620141E+01 max= 0.0000000000000000E+00 v MOM_grid_init/set_grid_metrics: geoLatC[uv]
v-point: c= 1850760 sw= 1861560 se= 1861560 nw= 1855800 ne= 1855800 v MOM_grid_init/set_grid_metrics: geoLatC[uv]
h-point: mean= 1.8742828447618181E+03 min= 5.0000000000000000E-01 max= 5.4547183562354667E+03 MOM_initialize_fixed: depth
h-point: c= 1505221 sw= 1503001 se= 1503001 nw= 1503834 ne= 1503834 MOM_initialize_fixed: depth
h-point: mean= 7.1373456790123457E-01 min= 0.0000000000000000E+00 max= 1.0000000000000000E+00 MOM_initialize_fixed: mask2dT
h-point: c= 462500 MOM_initialize_fixed: mask2dT
u-point: mean= 7.0998456790123454E-01 min= 0.0000000000000000E+00 max= 1.0000000000000000E+00 u MOM_initialize_fixed: mask2dC[uv]
u-point: c= 460070 u MOM_initialize_fixed: mask2dC[uv]
v-point: mean= 7.0817901234567904E-01 min= 0.0000000000000000E+00 max= 1.0000000000000000E+00 v MOM_initialize_fixed: mask2dC[uv]
v-point: c= 458900 v MOM_initialize_fixed: mask2dC[uv]
B-point: mean= 7.0442901234567901E-01 min= 0.0000000000000000E+00 max= 1.0000000000000000E+00 MOM_initialize_fixed: mask2dBu
B-point: c= 456470 MOM_initialize_fixed: mask2dBu
B-point: mean= -5.1935394571824036E-05 min= -9.5278618879650738E-05 max= 0.0000000000000000E+00 MOM_initialize_fixed: f
B-point: c= 2122200 MOM_initialize_fixed: f
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 MOM_initialize_fixed: dF_dx
h-point: c= 0 MOM_initialize_fixed: dF_dx
h-point: mean= 2.0823624514094935E-11 min= 1.7287465519113497E-11 max= 2.2866325108382357E-11 MOM_initialize_fixed: dF_dy
h-point: c= 2192760 MOM_initialize_fixed: dF_dy
MOM_initialize_coord: Rlay 2.2753266928533503E+04 592
MOM_initialize_coord: g_prime 9.8766479591356156E+00 700
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u MOM_initialize_state [uv]
u-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 u MOM_initialize_state [uv]
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v MOM_initialize_state [uv]
v-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 v MOM_initialize_state [uv]
h-point: mean= 8.5194674761900828E+01 min= 1.0000000000000000E-10 max= 2.7094558579452141E+03 MOM_initialize_state: h
h-point: c= 30808523 sw= 31001201 se= 31001201 nw= 31012932 ne= 31012932 MOM_initialize_state: h
h-point: mean= 1.2016580834073947E+01 min= 2.9999999999993889E+00 max= 2.5000000000000306E+01 MOM_initialize_state: T
h-point: c= 41572800 sw= 41337000 se= 41337000 nw= 41349960 ne= 41349960 MOM_initialize_state: T
h-point: mean= 3.5000000000000000E+01 min= 3.5000000000000000E+01 max= 3.5000000000000000E+01 MOM_initialize_state: S
h-point: c= 5702400 sw= 5670720 se= 5670720 nw= 5670720 ne= 5670720 MOM_initialize_state: S
MOM domain decomposition
whalo = 4, ehalo = 4, shalo = 4, nhalo = 4
X-AXIS = 52 52 51 50 51 52 52
Y-AXIS = 45 45 45 45
u-point: mean= 3.2272025813692477E-02 min= 0.0000000000000000E+00 max= 9.9999999995788746E-01 u btcalc frhat[uv]
u-point: c= 33295431 W= 33295431 u btcalc frhat[uv]
v-point: mean= 3.2189955106621766E-02 min= 0.0000000000000000E+00 max= 9.9999999995589761E-01 v btcalc frhat[uv]
v-point: c= 33222478 S= 33222478 v btcalc frhat[uv]
h-point: mean= 8.5194674761900828E+01 min= 1.0000000000000000E-10 max= 2.7094558579452141E+03 btcalc h
h-point: c= 30808523 sw= 31001201 se= 31001201 nw= 31012932 ne= 31012932 btcalc h
h-point: mean= 1.3444135763876215E+01 min= 4.1003812634885524E+00 max= 2.3899545898174800E+01 Post extract_sfc SST
h-point: c= 1791000 sw= 1717698 se= 1717698 nw= 1718404 ne= 1718404 Post extract_sfc SST
h-point: mean= 3.5000000000000000E+01 min= 3.5000000000000000E+01 max= 3.5000000000000000E+01 Post extract_sfc SSS
h-point: c= 259200 sw= 248512 se= 248512 nw= 248512 ne= 248512 Post extract_sfc SSS
h-point: mean= 3.4118964005379208E-13 min= -1.8189894035458565E-12 max= 2.2737367544323206E-12 Post extract_sfc sea_lev
h-point: c= 373250 sw= 360147 se= 360127 nw= 359733 ne= 359697 Post extract_sfc sea_lev
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 Post extract_sfc Hml
h-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 Post extract_sfc Hml
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u Post extract_sfc SSU
u-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 u Post extract_sfc SSU
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v Post extract_sfc SSU
v-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 v Post extract_sfc SSU
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 Post extract_sfc frazil
h-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 Post extract_sfc frazil

WARNING from PE 0: MPP_OPEN: File ./CPU_stats opened WRONLY already exists!

WARNING from PE 0: MPP_OPEN: File ./CPU_stats opened WRONLY already exists!

u-point: mean= 6.2918894440132295E-02 min= 8.7761861854524664E-04 max= 9.9998475124111127E-02 u After set forcing forces%tau[xy]
u-point: c= 2214720 W= 2214720 u After set forcing forces%tau[xy]
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v After set forcing forces%tau[xy]
v-point: c= 0 S= 0 v After set forcing forces%tau[xy]
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing forces%p_surf
h-point: c= 0 After set forcing forces%p_surf
h-point: mean= 8.7602643979869162E-03 min= 4.4912817983100498E-03 max= 1.0767569627147516E-02 After set forcing fluxes%ustar
h-point: c= 2146680 After set forcing fluxes%ustar
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%sw
h-point: c= 0 After set forcing fluxes%sw
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%lw
h-point: c= 0 After set forcing fluxes%lw
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%latent
h-point: c= 0 After set forcing fluxes%latent
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%latent_evap_diag
h-point: c= 0 After set forcing fluxes%latent_evap_diag
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%latent_fprec_diag
h-point: c= 0 After set forcing fluxes%latent_fprec_diag
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%latent_frunoff_diag
h-point: c= 0 After set forcing fluxes%latent_frunoff_diag
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%sens
h-point: c= 0 After set forcing fluxes%sens
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%evap
h-point: c= 0 After set forcing fluxes%evap
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%lprec
h-point: c= 0 After set forcing fluxes%lprec
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%fprec
h-point: c= 0 After set forcing fluxes%fprec
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%vprec
h-point: c= 0 After set forcing fluxes%vprec
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%seaice_melt
h-point: c= 0 After set forcing fluxes%seaice_melt
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%p_surf
h-point: c= 0 After set forcing fluxes%p_surf
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%lrunoff
h-point: c= 0 After set forcing fluxes%lrunoff
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%frunoff
h-point: c= 0 After set forcing fluxes%frunoff
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%heat_content_lrunoff
h-point: c= 0 After set forcing fluxes%heat_content_lrunoff
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%heat_content_frunoff
h-point: c= 0 After set forcing fluxes%heat_content_frunoff
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%heat_content_lprec
h-point: c= 0 After set forcing fluxes%heat_content_lprec
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%heat_content_fprec
h-point: c= 0 After set forcing fluxes%heat_content_fprec
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%heat_content_cond
h-point: c= 0 After set forcing fluxes%heat_content_cond
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 After set forcing fluxes%heat_content_massout
h-point: c= 0 After set forcing fluxes%heat_content_massout

WARNING from PE 0: MPP_OPEN: File ./ocean.stats opened WRONLY already exists!

WARNING from PE 0: MPP_OPEN: File ./ocean.stats opened WRONLY already exists!

MOM Date 1/01/01 00:00:00 0: En 5.483130E-01, MaxCFL 0.00000, Mass 7.909733697458E+19, Salt 35.00000000000, Temp 5.06383031873
Total Energy: 4402CF0C41B7B49B 4.3370095656201052E+19
Total Mass: 7.9097336974577992E+19, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
Total Salt: 2.7684067941102295E+18, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
Total Heat: 1.5721018104275223E+24, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
Total age: 0.0000000000000000E+00 yr kg
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u Beginning of step_MOM [uv]
u-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 u Beginning of step_MOM [uv]
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v Beginning of step_MOM [uv]
v-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 v Beginning of step_MOM [uv]
h-point: mean= 8.5194674761900828E+01 min= 1.0000000000000000E-10 max= 2.7094558579452141E+03 Beginning of step_MOM h
h-point: c= 30808523 sw= 31001201 se= 31001201 nw= 31012932 ne= 31012932 Beginning of step_MOM h
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u Beginning of step_MOM [uv]h
u-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 u Beginning of step_MOM [uv]h
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v Beginning of step_MOM [uv]h
v-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 v Beginning of step_MOM [uv]h
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u Before steps [uv]
u-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 u Before steps [uv]
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v Before steps [uv]
v-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 v Before steps [uv]
h-point: mean= 8.5194674761900828E+01 min= 1.0000000000000000E-10 max= 2.7094558579452141E+03 Before steps h
h-point: c= 30808523 sw= 31001201 se= 31001201 nw= 31012932 ne= 31012932 Before steps h
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u Before steps [uv]h
u-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 u Before steps [uv]h
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v Before steps [uv]h
v-point: c= 0 sw= 0 se= 0 nw= 0 ne= 0 v Before steps [uv]h
u-point: mean= 6.2918894440132295E-02 min= 8.7761861854524664E-04 max= 9.9998475124111127E-02 u Before steps forces%tau[xy]
u-point: c= 2214720 W= 2214720 u Before steps forces%tau[xy]
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v Before steps forces%tau[xy]
v-point: c= 0 S= 0 v Before steps forces%tau[xy]
h-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 Before steps forces%p_surf
h-point: c= 0 Before steps forces%p_surf
u-point: mean= 4.5377314481066648E+01 min= 2.4396444961540895E-02 max= 1.7150342230693926E+02 u Kh_[uv]
u-point: c= 35257804 u Kh_[uv]
v-point: mean= 4.4836833530353495E+01 min= 2.9500707440092422E-02 max= 1.5855182947774423E+02 v Kh_[uv]
v-point: c= 35256631 v Kh_[uv]
u-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 u int_slope_[uv]
u-point: c= 0 u int_slope_[uv]
v-point: mean= 0.0000000000000000E+00 min= 0.0000000000000000E+00 max= 0.0000000000000000E+00 v int_slope_[uv]
v-point: c= 0 v int_slope_[uv]
h-point: mean= 8.5194674761900828E+01 min= 1.0000000000000000E-10 max= 2.7094558579452141E+03 thickness_diffuse_1 h
h-point: c= 30808523 sw= 31001201 se= 31001201 nw= 31012932 ne= 31012932 thickness_diffuse_1 h
h-point: mean= -3.1184810990541325E+02 min= -5.4547183562354667E+03 max= 3.5527136788005009E-13 thickness_diffuse_1 e
h-point: c= 53478758 sw= 53531920 se= 53531920 nw= 53510754 ne= 53510754 thickness_diffuse_1 e
h-point: mean= 1.2016580834073947E+01 min= 2.9999999999993889E+00 max= 2.5000000000000306E+01 thickness_diffuse T
h-point: c= 41572800 sw= 41337000 se= 41337000 nw= 41349960 ne= 41349960 thickness_diffuse T
h-point: mean= 3.5000000000000000E+01 min= 3.5000000000000000E+01 max= 3.5000000000000000E+01 thickness_diffuse S
h-point: c= 5702400 sw= 5670720 se= 5670720 nw= 5670720 ne= 5670720 thickness_diffuse S
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FECB7E4E5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom_ 672 MOM.F90
MOM6 00000000007C1EE0 MAIN__ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FECB779BC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FCBC1A9A5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FCBC13E7C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F82A06235E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F829FF70C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FB5C7EDD5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FB5C782AC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FA30BCE35E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FA30B630C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007EFCD22705E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007EFCD1BBDC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FF25FAEC5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FF25F439C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F9DBBE475E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F9DBB794C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F541A96D5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F541A2BAC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F9EF7A5E5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F9EF73ABC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FC64BCCD5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FC64B61AC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F1236A535E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F12363A0C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FCCACE6D5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FCCAC7BAC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F7F258BA5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F7F25207C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FAD4A6A55E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FAD49FF2C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F69469015E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F694624EC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F2F982C65E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F2F97C13C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F81B99D55E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F81B9322C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FCAEF3AA5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FCAEECF7C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FC64FAC85E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FC64F415C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007FE86BEBC5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007FE86B809C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F2F664B15E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F2F65DFEC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F12F3D345E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F12F3681C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F1E1C8205E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F1E1C16DC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F7DFE7DB5E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F7DFE128C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F380DF645E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F380D8B1C05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007F697BC025E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007F697B54FC05 libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
MOM6 000000000117F4D4 Unknown Unknown Unknown
libpthread-2.17.s 00007EFCA72105E0 Unknown Unknown Unknown
MOM6 0000000000779E2B mom_thickness_dif 536 MOM_thickness_diffuse.F90
MOM6 0000000000765AF5 mom_thickness_dif 311 MOM_thickness_diffuse.F90
MOM6 0000000000B011EC mom_mp_step_mom_d 884 MOM.F90
MOM6 0000000000AF3B94 mom_mp_step_mom 672 MOM.F90
MOM6 00000000007C1EE0 MAIN_ 507 MOM_driver.F90
MOM6 000000000040C89E Unknown Unknown Unknown
libc-2.17.so 00007EFCA6B5DC05 __libc_start_main Unknown Unknown
MOM6 000000000040C7A9 Unknown Unknown Unknown

After setting ulimit -s unlimited, I am ending up with:

v-point: c= 33222397 S= 33222397 v btcalc frhat[uv]
u-point: mean= 8.5074641257582826E+01 min= 0.0000000000000000E+00 max= 2.7092621133018024E+03 u btcalc h_[uv]
u-point: c= 37011342 W= 37011342 u btcalc h_[uv]
v-point: mean= 8.5055807009653520E+01 min= -2.5315756865150547E+01 max= 2.8150631502617102E+03 v btcalc h_[uv]
v-point: c= 35406179 S= 35406179 v btcalc h_[uv]
h-point: mean= 8.5194674761900828E+01 min= 1.0000000000000000E-10 max= 2.7094552603333541E+03 btcalc h
h-point: c= 45524222 sw= 45705458 se= 45705458 nw= 45697729 ne= 45697729 btcalc h

FATAL from PE 0: NaN detected: u Predictor [uv]

FATAL from PE 0: NaN detected: u Predictor [uv]

application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
@vineetmoraybfab vineetmoraybfab changed the title Getting Getting Segmentation Fault for ocean_only/benchmark May 9, 2018
@Hallberg-NOAA
Copy link
Member

Thank you for letting us know, @boneles . Three further details would be helpful for us.

  1. What version of the MOM6 code are you using? (I.e., what git-hash?)
  2. If you try using -O2 or -O0, are you getting the same behavior? (In the past few years we have sometimes found compiler problems with overly aggressive optimization settings; the various compiler development groups are not so well funded as they once were.)
  3. How many processors were you using?

@vineetmoraybfab
Copy link
Author

Thanks for the suggestion Hallberg.
And yes, I got the problem fixed. It was (as you said) a compiler issue.

I accidently compiled the application using intel -xHost flag which compiled it for my cluster's login node.
Afterwards when I compiled using -axCORE-AVX2 flag(my compute node proc), this issue was no more.

Working like a charm...
Will try for KNL arch (hope for more performance) and benchmark the same. Thanks...

And sorry for marking an issue that was not actually a MOM issue.
Please close this ticket.

@Hallberg-NOAA
Copy link
Member

Thanks for letting us know, @boneles . Compiler bugs are rare (especially compared with user error), but they do happen!

marshallward added a commit that referenced this issue Apr 6, 2023
@marshallward
MOM6: Merge branch 'main' into merge_main_20230405
387ddb5 
- NOAA-GFDL/MOM6@fc823f5d7 Merge branch 'main' into merge_main_20230405
- NOAA-GFDL/MOM6@a6f813e05 (*)Fix MOM_calc_grad_Coriolis GLOBAL_INDEXING bug
- NOAA-GFDL/MOM6@1e54bed6a Merge pull request #1594 from NCAR/dev-ncar-main-candidate-2023-03-02
- NOAA-GFDL/MOM6@d1d53bcb4 remove inadvertent reassignment of layer_frac
- NOAA-GFDL/MOM6@ed93a232a  add SMOOTH_RI to MOM_obsolete_params.F90
- NOAA-GFDL/MOM6@774074ff9 minor styling fixes to address PR #1594 reviews.
- NOAA-GFDL/MOM6@9786710c3 Merge pull request #235 from mom-ocean/main
- NOAA-GFDL/MOM6@c142e4c39 Merge pull request #231 from jedwards4b/cesm_logging_improvements
- NOAA-GFDL/MOM6@4959ee12f update more write units to stdout in mom6 nuopc cap and let only rootpe write logs.
- NOAA-GFDL/MOM6@b693dd2d8 remove more direct calls to FMS in mom_cap.F90
- NOAA-GFDL/MOM6@1dce6df13 merge latest dev/ncar and resolve conflicts
- NOAA-GFDL/MOM6@5b274bbde Merge pull request #234 from alperaltuntas/fix_gnu_restart
- NOAA-GFDL/MOM6@0f1fc0386 fix gnu restart issue by removing leading spaces in restartfiles list
- NOAA-GFDL/MOM6@9d5d9bb40 Merge pull request #233 from marshallward/siglongjmp_name_fix
- NOAA-GFDL/MOM6@de8023d10 POSIX: siglongjmp and sigsetjmp_missing fixes
- NOAA-GFDL/MOM6@7f024685f Merge pull request #229 from gustavo-marques/merge_main_08Dec22
- NOAA-GFDL/MOM6@a789254e0 get the variable name correct
- NOAA-GFDL/MOM6@8bc69f67e further improvement to logunit setting
- NOAA-GFDL/MOM6@cac67d1e7 revert change to github action
- NOAA-GFDL/MOM6@12238ae21 remove whitespace
- NOAA-GFDL/MOM6@7824dcec7 switch to openmpi for github tests
- NOAA-GFDL/MOM6@a80b91ded use more cesm style logging
- NOAA-GFDL/MOM6@380864138 Switch from mpich to openmpi
- NOAA-GFDL/MOM6@ed252790d Merge branch 'dev/ncar' into main_08Dec22
- NOAA-GFDL/MOM6@1eb6be9c8 Merge pull request #222 from NCAR/willy_tracer
- NOAA-GFDL/MOM6@4840c6fdb Merge pull request #226 from gustavo-marques/fix_diags_docs
- NOAA-GFDL/MOM6@d5bd437b7 Merge pull request #225 from gustavo-marques/multiple_ri_smooth
- NOAA-GFDL/MOM6@0cf34d352 Fix string order in regional_section
- NOAA-GFDL/MOM6@b4440417c Adds option to smooth gradient Ri multiple times
- NOAA-GFDL/MOM6@09278521c Modify gradient Richarson number diagnostics
- NOAA-GFDL/MOM6@cfd7c0bde Rename parameter SMOOTH_RI to N_SMOOTH_RI
- NOAA-GFDL/MOM6@fe3cb4d69 Merge pull request #224 from gustavo-marques/merge-dev-gfdl-candidate-main-2022-08-10
- NOAA-GFDL/MOM6@36487d828 Merge branch 'main' into merge-dev-gfdl-candidate-main-2022-08-10
- NOAA-GFDL/MOM6@0b5cd6ff7 Merge pull request #223 from mom-ocean/main
- NOAA-GFDL/MOM6@dfb37154a Changed parameter reference for ideal_age_physics to Hbl=Hml
- NOAA-GFDL/MOM6@e300296ac Changed GV%nkbl back to GV%nkml
- NOAA-GFDL/MOM6@5ac624241 All references to "mixed layer" in the ideal age module now refer to "boundary layer" instead.
- NOAA-GFDL/MOM6@0d86acd52 Shortened line length to make dOxygen happy
- NOAA-GFDL/MOM6@2428684ac !! ---->  !<
- NOAA-GFDL/MOM6@f415a7f5c Shortens a line and removes whitespace
- NOAA-GFDL/MOM6@bb02a51be A new tracer that keeps track of "mixed layer age" has been added to the ideal age module. This PR also adds the ability to use the actual BL depth that is diagnosed by the active BL scheme inside the ideal age module (for all ideal age tracers).
- NOAA-GFDL/MOM6@d9ec82e4d Merge branch 'mom-ocean:main' into dev/ncar
- NOAA-GFDL/MOM6@605771423 Follow MOM6 code style guide
- NOAA-GFDL/MOM6@8e833d4d9 Add missing units
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Hallberg-NOAA @vineetmoraybfab