CICE Segfault with current GEOSgcm main

[mathomp4]

There seems to be a problem with GEOSgcm and CICE4 at the moment. After the latest rounds of updates to GEOSgcm, I am now seeing a segfault with CICE4 in both MOM5 and MOM6 runs. Both point to the same line (truncated traceback):

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
GEOSgcm.x          0000000003287CAA  Unknown               Unknown  Unknown
libpthread-2.22.s  00002AAAC9542C10  Unknown               Unknown  Unknown
GEOSgcm.x          0000000003025872  ice_flux_mp_get_o        2050  ice_flux.F90
GEOSgcm.x          0000000002FA5601  geos_cicedynagrid        2394  GEOS_CICEDynaGridComp.F90
libMAPL.generic.s  00002AAABF880D24  mapl_genericmod_m        1836  MAPL_Generic.F90
libMAPL.generic.s  00002AAABF88279D  mapl_genericmod_m        1953  MAPL_Generic.F90
GEOSgcm.x          0000000002F876E0  geos_seaicegridco         807  GEOS_SeaIceGridComp.F90
libMAPL.generic.s  00002AAABF880D24  mapl_genericmod_m        1836  MAPL_Generic.F90
libMAPL.generic.s  00002AAABF88279D  mapl_genericmod_m        1953  MAPL_Generic.F90
GEOSgcm.x          0000000002F789D9  geos_ogcmgridcomp        2069  GEOS_OgcmGridComp.F90
libMAPL.generic.s  00002AAABF880D24  mapl_genericmod_m        1836  MAPL_Generic.F90

Now, as far as I can tell, CICE4 hasn't changed in a long time, so this is probably on me. Two things have changed that I am suspecting:

1. We are using Intel 2021.2
2. We are using Vectorized flags everywhere

I'm going to try undoing #2 in CICE4. That's my best bet.

CC @sanAkel @yvikhlya @zhaobin74

[mathomp4]

Okay. I can fix the CICE4 failure by reverting the flags there and I'm going to work on getting that in.

But we have another issue: MOM5 and MOM6 are non-zero-diff now. I've tried reverting the build flags for MOM5 and MOM6 to the old flags as well but that doesn't make them zero-diff. So, I'm thinking this might be due to Intel 2021.2. If so...not much I can do.

[mathomp4]

Note: MOM5 and MOM6 are zero-diff with 1moment, just not 2moment. Thus I must assume this is the 2moment issue like we saw in MAPL: GEOS-ESM/MAPL#762

Not sure what triggered it, but I think day of the week can trigger non-zero-diff with 2moment it seems.

[yvikhlya]

Note: MOM5 and MOM6 are zero-diff with 1moment, just not 2moment. Thus I must assume this is the 2moment issue like we saw in MAPL: GEOS-ESM/MAPL#762

Not sure what triggered it, but I think day of the week can trigger non-zero-diff with 2moment it seems.

Matt, thanks you are looking into this. For coupled model we are trying to run different microphysics in feature/aogcm branch by the way, it may have different problems.

[mathomp4]

Matt, thanks you are looking into this. For coupled model we are trying to run different microphysics in feature/aogcm branch by the way, it may have different problems.

Eeep! Well, we shall see when we get to it!