FVCOM 4.4.7 with PETSc 3.18.5 build syntax errors (f951: Error: Unexpected end of file in ‘mod_petsc.f90’) on Rocky Linux 9

[danesnick]

Hi,

I'm attempting to build FVCOM 4.4.7 on our local HPC cluster using GCC 11.3 and MPICH 4.2 running Rocky Linux 9, I had to make some config changes to get the libraries built, and then run into a final snag at the end of the build process.

First, here's the build environment:

08:21:21  |makedepf90_env|ndanes@wendian001 scratch → echo $PATH
/sw/mpi/mpich/4.2.1/gcc/bin:/opt/rh/devtoolset-11/root/usr/bin:/projects/shared_conda_envs/envs/makedepf90_env/bin:/sw/apps/python3/anaconda-2023.07/condabin:/sw/apps/python3/anaconda-2023.07/bin:/wendianSoftware/x86_64/sw/utility/spack/0.22.0/bin:/u/pa/sh/ndanes/.local/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/u/pa/sh/ndanes/bin
08:21:23  |makedepf90_env|ndanes@wendian001 scratch → gcc --version
gcc (GCC) 11.4.1 20231218 (Red Hat 11.4.1-3)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

08:21:29  |makedepf90_env|ndanes@wendian001 scratch → mpirun --version
HYDRA build details:
    Version:                                 4.2.1
    Release Date:                            Wed Apr 17 15:30:02 CDT 2024
    CC:                              gcc
    Configure options:                       '--disable-option-checking' '--prefix=/sw/mpi/mpich/4.2.1/gcc' '--with-hwloc=embedded' '--with-slurm-libdir=/usr/lib64/slurm' '--with-pmi-libdir=/usr/lib64/slurm' '--enable-shared' '--enable-mpi-cxx' '--with-ucx=embedded' 'CC=gcc' 'CXX=g++' 'FC=gfortran' '--with-device=ch4:ucx' 'FFLAGS= -O2' '--cache-file=/dev/null' '--srcdir=.' 'CFLAGS= -O2' 'LDFLAGS=' 'LIBS=' 'CPPFLAGS= -DNETMOD_INLINE=__netmod_inline_ucx__ -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/src/mpl/include -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/modules/json-c -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/modules/hwloc/include -D_REENTRANT -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/src/mpi/romio/include -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/src/pmi/include -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/modules/yaksa/src/frontend/include -I/sw/BUILD/scratch/mpich/mpich-4.2.1-gcc/modules/ucx/src'
    Process Manager:                         pmi
    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist
    Topology libraries available:            hwloc
    Resource management kernels available:   user slurm ll lsf sge pbs cobalt
    Demux engines available:                 poll select

Here are the relevant bits of my $TOPDIR/make.inc file:

         CPP      = /usr/bin/cpp
         COMPILER = -DGFORTRAN
         CC       = mpicc
         CXX      = mpicxx
         CFLAGS   = -O3
         FC       = mpif90
         DEBFLGS  =
         OPT      = -O3

#        If FLAG_411 is defined, uncomment path below
             PARLIB =
             PARTINCS = -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include
             PARTLIBS = -L/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/lib -lmetis

            FLAG_41 = -DPETSC_C
            include ${PETSC_DIR}/lib/petsc/conf/variables

Then, I had to make a small modification with $TOPDIR/src/libs/makefile to get fproj to build by adding the -Df2cFortran in my CFLAGS:

 cd fproj && ./configure CPPFLAGS='$(COMPILER)' CC=$(CC) CFLAGS=' -Df2cFortran -O3' CXX=$(CXX) CXXFLAGS=-O3 FC=$(FC) FFLAGS=-O3 --with-proj4=$(MYINSTALLDIR) --prefix=$(MYINSTALLDIR)

The $TOPDIR/src/libs/makefile build fine with this configuration. PETSc 3.18.5 was configured just using:

./configure --download-metis --download-parametis --with-fc=$(which mpif90) --with-cc=$(which mpicc) --with-cxx=$(which mpicxx)

Now when I go back to $TOPDIR/src to build the main program, most stuff completes until I hit the end:

mpif90  -c  -O3 -I /u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/netcdf/f90     -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/julian -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include mod_petsc.f90
mod_petsc.f90:841:132:

  841 | ETSC_COMM_SELF,BS,PSIZE_EL_HALO,PTMP,PETSC_COPY_VALUES,ISL2G_EL_col,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,588,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:842:132:

  842 | ETSC_COMM_SELF,BS,PSIZE_EL_HALO,PTMP,PETSC_COPY_VALUES,ISL2G_EL_row,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,589,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:857:132:

  857 | L,ISL2G_EL_row,ISL2G_EL_col,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,630,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:857:132:

  857 | L,ISL2G_EL_row,ISL2G_EL_col,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,630,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Missing ‘)’ in statement at or before (1)
mod_petsc.f90:857:132:

  857 | L,ISL2G_EL_row,ISL2G_EL_col,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,630,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Syntax error in argument list at (1)
mod_petsc.f90:869:132:

  869 | ,N_VERTS,0,1,ISLOCAL_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,651,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:874:132:

  874 | ,N_VERTS,ACCUM(MYID),1,ISGLOBAL_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,665,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:874:132:

  874 | ,N_VERTS,ACCUM(MYID),1,ISGLOBAL_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,665,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Missing ‘)’ in statement at or before (1)
mod_petsc.f90:874:128:

  874 | SELF,N_VERTS,ACCUM(MYID),1,ISGLOBAL_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,665,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Unterminated character constant beginning at (1)
mod_petsc.f90:904:132:

  904 | _EL,X_EL,ISGLOBAL_EL,L2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,728,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:904:132:

  904 | _EL,X_EL,ISGLOBAL_EL,L2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,728,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Missing ‘)’ in statement at or before (1)
mod_petsc.f90:904:132:

  904 | _EL,X_EL,ISGLOBAL_EL,L2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,728,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Syntax error in argument list at (1)
mod_petsc.f90:909:132:

  909 | _EL,XL_EL,ISLOCAL_EL,G2L_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,741,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:909:132:

  909 | _EL,XL_EL,ISLOCAL_EL,G2L_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,741,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Missing ‘)’ in statement at or before (1)
mod_petsc.f90:909:132:

  909 | _EL,XL_EL,ISLOCAL_EL,G2L_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,741,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Syntax error in argument list at (1)
mod_petsc.f90:929:132:

  929 | sNonzero(Ksp_EL,PETSC_TRUE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,782,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:958:5:

  958 |   END SUBROUTINE PETSc_SET
      |     1
Error: Expecting END IF statement at (1)
mod_petsc.f90:522:6:

  522 |   USE ALL_VARS, ONLY : NVG, MSR, NGL, MGL, MT, NPROCS, MYID, KBM1, M, IPT
      |      1
......
  979 |   USE ALL_VARS, ONLY : M, KBM1, NTSN, NBSN, MGL
      |                        2
Error: Symbol ‘m’ at (1) conflicts with the symbol at (2)
mod_petsc.f90:980:38:

  980 |   USE CONTROL,  ONLY : MPI_FVCOM_GROUP
      |                                      1
Error: Unexpected USE statement at (1)
mod_petsc.f90:981:15:

  981 |   IMPLICIT NONE
      |               1
Error: Duplicate IMPLICIT NONE statement at (1)
mod_petsc.f90:982:64:

  982 |   INTEGER :: NODE,LAY,PETSc_POS,NCOL,NCOL2,NEYNUM,NEY,COLCOUNT,I
      |                                                                1
Error: Symbol ‘i’ at (1) already has basic type of INTEGER
mod_petsc.f90:983:29:

  983 |   INTEGER :: COL2(MAX_NZ_ROW)
      |                             1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:984:34:

  984 |   INTEGER,ALLOCATABLE  :: ONNZ2(:)
      |                                  1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:985:34:

  985 |   INTEGER,ALLOCATABLE  :: DNNZ2(:)
      |                                  1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:986:33:

  986 |   CHARACTER(LEN=20)    :: SUBNAME = 'PETSc_AllocA'
      |                                 1
Error: Symbol ‘subname’ at (1) already has basic type of CHARACTER
mod_petsc.f90:989:44:

  989 |   integer(kind=selected_int_kind(5)) :: IERR
      |                                            1
Error: Symbol ‘ierr’ at (1) already has basic type of INTEGER
mod_petsc.f90:998:11:

  998 |   ALLOCATE(ONNZ2(N_VERTS)) ; ONNZ2 = 0
      |           1
Error: Allocate-object at (1) is neither a data pointer nor an allocatable variable
mod_petsc.f90:999:11:

  999 |   ALLOCATE(DNNZ2(N_VERTS)) ; DNNZ2 = 0
      |           1
Error: Allocate-object at (1) is neither a data pointer nor an allocatable variable
mod_petsc.f90:1024:16:

 1024 |         IF(COL2(COLCOUNT) <= N_VERTS) THEN
      |                1
Error: Syntax error in IF-expression at (1)
mod_petsc.f90:1026:12:

 1026 |         ELSE
      |            1
Error: Unexpected ELSE statement at (1)
mod_petsc.f90:1028:11:

 1028 |         ENDIF
      |           1
Error: Expecting END DO statement at (1)
mod_petsc.f90:1038:132:

 1038 | _VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,1042,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:1038:132:

 1038 | _VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,1042,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Missing ‘)’ in statement at or before (1)
mod_petsc.f90:1038:132:

 1038 | _VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,1042,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Syntax error in argument list at (1)
mod_petsc.f90:1039:132:

 1039 | ZERO_ALLOCATION_ERR, PETSC_FALSE, IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,1043,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Line truncated at (1) [-Werror=line-truncation]
mod_petsc.f90:1039:132:

 1039 | ZERO_ALLOCATION_ERR, PETSC_FALSE, IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,1043,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Missing ‘)’ in statement at or before (1)
mod_petsc.f90:1039:128:

 1039 | _NONZERO_ALLOCATION_ERR, PETSC_FALSE, IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,1043,"mod_petsc.F");return;endif
      |                                                                                               1

Error: Unterminated character constant beginning at (1)
mod_petsc.f90:1043:19:

 1043 |   DEALLOCATE(ONNZ2,DNNZ2)
      |                   1
Error: Allocate-object at (1) is not a nonprocedure pointer nor an allocatable variable
mod_petsc.f90:1048:5:

 1048 |   END SUBROUTINE PETSc_AllocA
      |     1
Error: Expecting END IF statement at (1)
mod_petsc.f90:1053:15:

 1053 |   IMPLICIT NONE
      |               1
Error: Duplicate IMPLICIT NONE statement at (1)
mod_petsc.f90:1054:52:

 1054 |   integer(kind=selected_int_kind(5)) :: REASON, IERR
      |                                                    1
Error: Symbol ‘ierr’ at (1) already has basic type of INTEGER
mod_petsc.f90:1074:5:

 1074 |   END SUBROUTINE PETSc_SOLVER_EL
      |     1
Error: Expecting END IF statement at (1)
mod_petsc.f90:1084:15:

 1084 |   IMPLICIT NONE
      |               1
Error: Duplicate IMPLICIT NONE statement at (1)
mod_petsc.f90:1085:44:

 1085 |   integer(kind=selected_int_kind(5)) :: IERR
      |                                            1
Error: Symbol ‘ierr’ at (1) already has basic type of INTEGER
mod_petsc.f90:1102:5:

 1102 |   END SUBROUTINE PETSc_CLEANUP
      |     1
Error: Expecting END IF statement at (1)
mod_petsc.f90:1114:15:

 1114 |   IMPLICIT NONE
      |               1
Error: Duplicate IMPLICIT NONE statement at (1)
mod_petsc.f90:1115:29:

 1115 |   INTEGER, INTENT(IN) :: NODE
      |                             1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:1116:28:

 1116 |   INTEGER, INTENT(IN) :: LAY
      |                            1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:1117:41:

 1117 |   INTEGER             :: PLO_LOC,PLO_NODE
      |                                         1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:1123:5:

 1123 |   END FUNCTION ALO_2_PLO
      |     1
Error: Expecting END IF statement at (1)
mod_petsc.f90:1136:15:

 1136 |   IMPLICIT NONE
      |               1
Error: Duplicate IMPLICIT NONE statement at (1)
mod_petsc.f90:1137:32:

 1137 |   INTEGER, INTENT(IN) :: PLO_LOC
      |                                1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:1138:35:

 1138 |   INTEGER             :: ALO_LOC(2)
      |                                   1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:1139:42:

 1139 |   INTEGER             :: LAY,NODE,PLO_NODE
      |                                          1
Error: Unexpected data declaration statement at (1)
mod_petsc.f90:1152:5:

 1152 |   END FUNCTION PLO_2_ALO
      |     1
Error: Expecting END IF statement at (1)
mod_petsc.f90:1158:3:

 1158 | END MODULE MOD_PETSc
      |   1
Error: Expecting END IF statement at (1)
f951: Error: Unexpected end of file in ‘mod_petsc.f90’
f951: some warnings being treated as errors
make: *** [makefile:45: mod_petsc.o] Error 1

[Pbarbosa92]

It turns out Fortran has a line length limit. Everything above that limit will be ignored.
Try adding -ffixed-line-length-none -ffree-line-length-0 to your mpif90 flags.
In my case I was using gfortran and I had to edit make.inc to have this:

#  gfortran defs
#--------------------------------------------------------------------------
         CPP      = /usr/bin/cpp
         COMPILER = -DGFORTRAN
         FC       = gfortran
         DEBFLGS  = -O3
         OPT      = -ffixed-line-length-none -ffree-line-length-0
         CLIB     =

The lines were no longer being truncated and that solved the issue.

Hope it helps.

[danesnick]

Hey! Thanks I totally got tunnel vision on the error message here. I'll give the flag a try and report back.

[danesnick]

New error:

/usr/bin/cpp -P -traditional  -DGFORTRAN   -P -traditional    -DWET_DRY -DMULTIPROCESSOR   -DDATA_ASSIM -DLIMITED_NO -DSEMI_IMPLICIT -DGCN    -DRIVER_FLOAT -DMPDATA             -DAIR_PRESSURE               -DPETSC_C   -DTVD   -DMETIS_5               -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include -I /u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/netcdf/f90     -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/julian -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include mod_station_timeseries.F > mod_station_timeseries.f90
mpif90  -c  -O3 -ffixed-line-length-none -ffree-line-length-0 -I /u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/netcdf/f90     -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/julian -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/include -I/u/pa/sh/ndanes/scratch/FVCOM-release-v4.4.7/src/libs/petsc-3.18.5/gcc/include mod_station_timeseries.f90
mod_station_timeseries.f90:1130:25:

 1130 |          WRITE(10,'(2I10,<MX_NBR_ELEM_GL+1>I10)') I,NTVEGL(I),(NBVEGL(I,J),J=1,NTVEGL(I))
      |                         1
Error: Unexpected element ‘<’ in format string at (1)

A quick google search seems to suggest that this line was written with the intent of a Intel Fortran compiler-only feature:

https://stackoverflow.com/questions/10509734/gfortran-error-unexpected-element-in-format-string

[mcarvajalino]

Oh I had to solve that problem with "unexpected element '<' just yesterday ... Try modifying:
<MX_NBR_ELEM_GL+1>I10
With
*(I10)
In mod_station_timeseries.f90

You might (like me) get a new error at the end of the compilation about libjulian not supporting PIE when the makefile tries to make the fvcom executable. To solve this you need to remake libjulian using the cflag -no-pie (add it in JOPTS).

I got it to compile like that ... Although now the executable fails because it cannot find libifports... That's tomorrows battle :)

[danesnick]

Thanks for the update! I'll see if I can get further in the build with this info.

[danesnick]

After adding -fallow-argument-mismatch to address some rank mismatching, and modifying '==' signs in with .eq. and .eqv. (depending on the line) in ocpcre.F, and adding hypre + fblaslapack to PETSC, I got libfvcom.a to build! I need to test still but I'll update you.

[riquitorres]

Hi Danesnick, Marcos,
We have also upgraded our HPC and are trying to compile FVCOM with a newer PETSC and not having a lot of joy!. I installed the latest 3.21.5 and Danesnick versions 3.18.5. We have mpiifx version ifx (IFX) 2024.1.0 20240308. I am compiling FVCOM with swave and semi-implicit. My first errors related to definition of variables. the fortran include files for variable definitions are in a different location so I ammended the include statements in the top of mod_petsc.F and had to add USE statements

USE petscmat
USE petscksp
USE petscpc
IMPLICIT NONE
#include "petsc/finclude/petsc.h"
#include "petsc/finclude/petscvec.h"
#include "petsc/finclude/petscdmda.h"
#include "petsc/finclude/petscmat.h"
#include "petsc/finclude/petscksp.h"
#include "petsc/finclude/petscpc.h"
#include "petsc/finclude/petscis.h"
#include "petsc/finclude/petscao.h"
#include "petsc/finclude/petscvec.h"
#include "petsc/finclude/petscviewer.h"

However the arguments have changed or the compiler checks are more stringent... before i try to figure out how to update the code I was wondering if you had to do anything similar. I don't see any changes in mot_petsc.F from version 4.4.7 to v5.0 or 4.3 which is the one I am using.

Example of the errors I get:
mpiifx -c -O3 -qno-openmp-simd -Ofast -march=core-avx2 -mtune=core-avx2 -I/scyllapfs/hpe/fthomas/css/.local/netcdf-4.9.0/oneapi/include -I/users/modellers/rito/Code/lakeerie/code/FVCOM_source/code/FVCOM_source/libs/install/include -I/users/modellers/rito/Code/lakeerie/code/FVCOM_source/install/fabm/include -I//users/modellers/rito/Code/lakeerie/code/FVCOM_source/install/fabm/include/yaml -I/work/rito/apps/petsc/intelmpi/3.18.5/include mod_petsc.f90
mod_petsc.f90(888): error #6633: The type of the actual argument differs from the type of the dummy argument. [ISL2G_EL]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_EL_HALO,PTMP,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,568,"mod_petsc.F");return;endif
---------------------------------------------------------------------^
mod_petsc.f90(888): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [F]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_EL_HALO,PTMP,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,568,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(888): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [Z]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_EL_HALO,PTMP,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,568,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(888): error #6634: The shape matching rules of actual arguments and dummy arguments have been violated. [PTMP]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_EL_HALO,PTMP,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,568,"mod_petsc.F");return;endif
----------------------------------------------------------------^
mod_petsc.f90(888): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [D]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_EL_HALO,PTMP,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,568,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(896): error #6633: The type of the actual argument differs from the type of the dummy argument. [ISL2G_WAVE]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_WAVE_HALO,PTMP,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,578,"mod_petsc.F");return;endif
-----------------------------------------------------------------------^
mod_petsc.f90(896): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [F]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_WAVE_HALO,PTMP,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,578,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(896): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [Z]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_WAVE_HALO,PTMP,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,578,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(896): error #6634: The shape matching rules of actual arguments and dummy arguments have been violated. [PTMP]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_WAVE_HALO,PTMP,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,578,"mod_petsc.F");return;endif
------------------------------------------------------------------^
mod_petsc.f90(896): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [D]
CALL ISLocalToGlobalMappingCreate(MPI_COMM_SELF,PSIZE_WAVE_HALO,PTMP,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,578,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(910): warning #6075: The data type of the actual argument does not match the definition. [IERR]
CALL MatSetLocalToGlobalMapping(A_EL,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,604,"mod_petsc.F");return;endif
------------------------------------------------^
mod_petsc.f90(910): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [Z]
CALL MatSetLocalToGlobalMapping(A_EL,ISL2G_EL,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,604,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(912): warning #6075: The data type of the actual argument does not match the definition. [IERR]
CALL MatSetLocalToGlobalMapping(A_WAVE,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,608,"mod_petsc.F");return;endif
----------------------------------------------------^
mod_petsc.f90(912): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [Z]
CALL MatSetLocalToGlobalMapping(A_WAVE,ISL2G_WAVE,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,608,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(988): error #6784: The number of actual arguments cannot be greater than the number of dummy arguments. [KSPSETOPERATORS]
CALL KSPSetOperators(Ksp_EL,A_EL,A_EL,SAME_NONZERO_PATTERN,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,746,"mod_petsc.F");return;endif
-------^
mod_petsc.f90(994): error #6404: This name does not have a type, and must have an explicit type. [PETSC_DEFAULT_DOUBLE_PRECISION]
CALL KSPSetTolerances(Ksp_EL,RTOL,PETSC_DEFAULT_DOUBLE_PRECISION, &
------------------------------------^
mod_petsc.f90(1000): error #6784: The number of actual arguments cannot be greater than the number of dummy arguments. [KSPSETOPERATORS]
CALL KSPSetOperators(Ksp_WAVE,A_WAVE,A_WAVE,DIFFERENT_NONZERO_PATTERN,IERR);if (IERR .ne. 0) then;call PetscErrorF(IERR,760,"mod_petsc.F");return;endif
-------^

[Pbarbosa92]

Did you change your make.inc file to use the newer version of petsc?

#       FOR VERSION 2.0   TO 2.3.2
#            FLAG_41 = -DPETSC_A
#            include ${PETSC_DIR}/bmake/common/variables
#
#       FOR VERSION 2.3.3
#            FLAG_41 = -DPETSC_B
#            include ${PETSC_DIR}/bmake/common/variables
#
#       FOR VERSION 3.18.5
            FLAG_41 = -DPETSC_C
            include ${PETSC_DIR}/lib/petsc/conf/variables

[riquitorres]

I did change the include path but not the Flag... I can see now that the mod_petsc was changed in version 5.1. I had only downloaded up to 5.0 and didn't see those changes in my local repository. Thank you for the quick response!
Having changed mod_petsc to the latest version it seems that the combination of semi-implicit and wave-current-interaction is not contemplated in the update... I will try to mimic the changes and hope for the best

[riquitorres]

After including the equivalent semi_implicit PETSC_C for the wave_current_interaction the last error I got was in the linker,

.....-Wl,-rpath,/work/rito/apps/petsc/intelmpi/3.18.5/lib -lpetsc -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lX11 -lstdc++ -ldl -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lpthread -lgcc_s -lirc_s -lstdc++ -ldl -o fvcom
ld: mod_petsc.o: in function mod_petsc_mp_petsc_set_': mod_petsc.f90:(.text+0x44b1): undefined reference to pchypresettype_'
pchypresettype is included in petscpc mod which is correctly invoked at the begining of mod_petsc.F
When I comment the culprit line
CALL PCHYPRESetType(Pc,"boomeramg",IERR);CHKERRQ(IERR)
the compilation proceeds to completion.
That same line is commented in the wave_current section (with a couple more lines) but not in the NH or semi-implicit sections.
I haven't tried to use the binary yet. If anyone can shed any light on this I would be greatful!
Regards
Ricardo

[mcarvajalino]

Hi Ricardo, Did you compile PETSC including hypre? This error seems to be pointing more to hypre than to petsc imho. Best, Marcos Best tor. 19. sep. 2024, 14:50 skrev Ricardo Torres ***@***.***>:

…

After including the equivalent semi_implicit PETSC_C for the wave_current_interaction the last error I got was in the linker, .....-Wl,-rpath,/work/rito/apps/petsc/intelmpi/3.18.5/lib -lpetsc -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lX11 -lstdc++ -ldl -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lpthread -lgcc_s -lirc_s -lstdc++ -ldl -o fvcom ld: mod_petsc.o: in function mod_petsc_mp_petsc_set_': mod_petsc.f90:(.text+0x44b1): undefined reference to pchypresettype_' pchypresettype is included in petscpc mod which is correctly invoked at the begining of mod_petsc.F When I comment the culprit line CALL PCHYPRESetType(Pc,"boomeramg",IERR);CHKERRQ(IERR) the compilation proceeds to completion. That same line is commented in the wave_current section (with a couple more lines) but not in the NH or semi-implicit sections. I haven't tried to use the binary yet. If anyone can shed any light on this I would be greatful! Regards Ricardo — Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4T34GKSZUZQFRDLPZDXY3ZXLCCLAVCNFSM6AAAAABMQPU2RKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRQHEYDCOBYHA> . You are receiving this because you commented.Message ID: ***@***.***>

[riquitorres]

that could very well be the case. I didn't add any options to the configuration and installation of PETSC. Will re-install with the option. Thank you! Ricardo

________________________________ From: Marcos Antonio Carvajalino Fernández ***@***.***> Sent: Thursday, September 19, 2024 14:12 To: FVCOM-GitHub/FVCOM ***@***.***> Cc: Ricardo Torres ***@***.***>; Comment ***@***.***> Subject: Re: [FVCOM-GitHub/FVCOM] FVCOM 4.4.7 with PETSc 3.18.5 build syntax errors (f951: Error: Unexpected end of file in ‘mod_petsc.f90’) on Rocky Linux 9 (Issue #26) Caution:This is an external email. Please take care when clicking links or opening attachments. When in doubt, contact your IT Department Hi Ricardo, Did you compile PETSC including hypre? This error seems to be pointing more to hypre than to petsc imho. Best, Marcos Best tor. 19. sep. 2024, 14:50 skrev Ricardo Torres ***@***.***>:

After including the equivalent semi_implicit PETSC_C for the wave_current_interaction the last error I got was in the linker, .....-Wl,-rpath,/work/rito/apps/petsc/intelmpi/3.18.5/lib -lpetsc -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lX11 -lstdc++ -ldl -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lpthread -lgcc_s -lirc_s -lstdc++ -ldl -o fvcom ld: mod_petsc.o: in function mod_petsc_mp_petsc_set_': mod_petsc.f90:(.text+0x44b1): undefined reference to pchypresettype_' pchypresettype is included in petscpc mod which is correctly invoked at the begining of mod_petsc.F When I comment the culprit line CALL PCHYPRESetType(Pc,"boomeramg",IERR);CHKERRQ(IERR) the compilation proceeds to completion. That same line is commented in the wave_current section (with a couple more lines) but not in the NH or semi-implicit sections. I haven't tried to use the binary yet. If anyone can shed any light on this I would be greatful! Regards Ricardo — Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4T34GKSZUZQFRDLPZDXY3ZXLCCLAVCNFSM6AAAAABMQPU2RKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRQHEYDCOBYHA> . You are receiving this because you commented.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<#26 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGHWP4CF2MZEYTEF6JK5FTZXLEVHAVCNFSM6AAAAABMQPU2RKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRQHE2TCMZYHE>. You are receiving this because you commented.Message ID: ***@***.***> Visit our website www.pml.ac.uk | Subscribe to our newsletter <https://www.pml.ac.uk/newsletter> | Follow us on social media: LinkedIn<https://www.linkedin.com/company/plymouth-marine-laboratory-&-pml-applications-ltd>, X / Twitter<https://twitter.com/PlymouthMarine>, Instagram<https://www.instagram.com/plymouthmarinelab>, Facebook<https://www.facebook.com/PlymouthMarineLaboratory>, YouTube<https://www.youtube.com/channel/UCfsYrcvGOp-JJm5owSJN8OA> This message is private and confidential. If you have received this message in error, please notify the sender and remove it from your system. You are reminded that e-mail communications are not secure and may contain viruses; PML accepts no liability for any loss or damage which may be caused by viruses. Our Privacy Notice is available at http://www.pml.ac.uk/System-pages/Privacy_Notice . Plymouth Marine Laboratory (PML) is a company limited by guarantee registered in England & Wales, company number 4178503. Registered Charity No. 1091222. Registered Office: Prospect Place, The Hoe, Plymouth PL1 3DH, UK. PML Applications Ltd, www.pml-applications.co.uk , is the wholly owned subsidiary of PML, registered as a company in England & Wales, company number 4232750.

[danesnick]

Just as an FYI @riquitorres , here's what I used to build PETSC 3.18 (GCC+MPICH)

./configure  --download-metis --download-parametis --with-fc=mpif90 --with-cc=mpicc --with-cxx=mpicxx --download-hypre --download-fblaslapack
make PETSC_DIR=$TOPDIR/libs/petsc-3.18.5 PETSC_ARCH=gcc all
make PETSC_DIR=$TOPDIR/libs/petsc-3.18.5 PETSC_ARCH=gcc check

[liesvyvall]

-no-pie

Hi, im having this error when compiling Julian:

mpif90 -O3 -no-pie -o tconvert tconvert.for libjulian.a
/usr/bin/ld: /tmp/cc1aLpZm.o: en la función MAIN__': tconvert.for:(.text+0x208): referencia a fjul_parsedt_' sin definir
/usr/bin/ld: tconvert.for:(.text+0x294): referencia a fjul_formatpds_' sin definir /usr/bin/ld: tconvert.for:(.text+0x33e): referencia a fjul_formatsql_' sin definir
/usr/bin/ld: tconvert.for:(.text+0x425): referencia a fjul_taiofdutc_' sin definir /usr/bin/ld: tconvert.for:(.text+0x443): referencia a fjul_etoftai_' sin definir
/usr/bin/ld: tconvert.for:(.text+0x694): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x710): referencia a fjul_jdoftai_' sin definir
/usr/bin/ld: tconvert.for:(.text+0x788): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x804): referencia a fjul_mjdoftai_' sin definir
/usr/bin/ld: tconvert.for:(.text+0x880): referencia a fjul_mjdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x8f8): referencia a fjul_mjdoftai_' sin definir

Did you know what's happening? thanks

[Pbarbosa92]

-no-pie

Hi, im having this error when compiling Julian:

mpif90 -O3 -no-pie -o tconvert tconvert.for libjulian.a /usr/bin/ld: /tmp/cc1aLpZm.o: en la función MAIN__': tconvert.for:(.text+0x208): referencia a fjul_parsedt_' sin definir /usr/bin/ld: tconvert.for:(.text+0x294): referencia a fjul_formatpds_' sin definir /usr/bin/ld: tconvert.for:(.text+0x33e): referencia a fjul_formatsql_' sin definir /usr/bin/ld: tconvert.for:(.text+0x425): referencia a fjul_taiofdutc_' sin definir /usr/bin/ld: tconvert.for:(.text+0x443): referencia a fjul_etoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x694): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x710): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x788): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x804): referencia a fjul_mjdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x880): referencia a fjul_mjdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x8f8): referencia a fjul_mjdoftai_' sin definir

Did you know what's happening? thanks

Hi,

This looks like a julian problem to me. Did you compile julian? You have julian (and other libraries inside /src/libs).
You can do make inside the libs folder to build all libraries needed (check the Makefile to see if you dont have some code commented).

After that:

• make sure you have the libs/isntall/include in your INCLUDEPATH
  • export INCLUDEPATH=/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/openmpi:/FVCOM/src/libs/install/include/:/usr/include/
• In your make.inc edit the DTLIBS to point to the libs install folder DTLIBS = -L/FVCOM/src/libs/install/lib -ljulian
• Go to the libs folder and compile them with make
  • Make sure all Packages (needed) are present in the makefile Ex: PACKAGES = proj fproj julian metis netcdf
  • For fproj I had to edit the CFLAGS (not sure if you will have to do this) cd fproj && ./configure CPPFLAGS='$(COMPILER)' CC=$(CC) CFLAGS="-O3 -Df2cFortran" CXX=$(CXX) CXXFLAGS=-O3 FC=$(FC) FFLAGS=-O3 --with-proj4=$(MYINSTALLDIR) --prefix=$(MYINSTALLDIR)
  • Install all by doing make
• Compile FVCOM

[mcarvajalino]

Hi liesvy It seems to me your compilation is not finding some functions (te dice "sin definir"). Are you compiling using the makefile in the Julian folder? That works ok for me Best Marcos søn. 22. sep. 2024, 05:10 skrev Liesvy Valladares ***@***.***

…

: -no-pie Hi, im having this error when compiling Julian: mpif90 -O3 -no-pie -o tconvert tconvert.for libjulian.a /usr/bin/ld: /tmp/cc1aLpZm.o: en la función MAIN__': tconvert.for:(.text+0x208): referencia a fjul_parsedt_' sin definir /usr/bin/ld: tconvert.for:(.text+0x294): referencia a fjul_formatpds_' sin definir /usr/bin/ld: tconvert.for:(.text+0x33e): referencia a fjul_formatsql_' sin definir /usr/bin/ld: tconvert.for:(.text+0x425): referencia a fjul_taiofdutc_' sin definir /usr/bin/ld: tconvert.for:(.text+0x443): referencia a fjul_etoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x694): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x710): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x788): referencia a fjul_jdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x804): referencia a fjul_mjdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x880): referencia a fjul_mjdoftai_' sin definir /usr/bin/ld: tconvert.for:(.text+0x8f8): referencia a fjul_mjdoftai_' sin definir Did you know what's happening? thanks — Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4T34EEGRSM24NCSBTM4P3ZXYYI3AVCNFSM6AAAAABMQPU2RKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRVGQZTMOJTGM> . You are receiving this because you commented.Message ID: ***@***.***>

[liesvyvall]

Thanks,

it seems that something I had changed in the code inside the libs. I copied it again, changed the line in fproj to: _cd fproj && ./configure CPPFLAGS='$(COMPILER)' CC=$(CC) CFLAGS="-O3 -Df2cFortran" CXX=$(CXX) CXXFLAGS=-O3 FC=$(FC) FFLAGS=-O3 --with-proj4=$(MYINSTALLDIR) --prefix=$(MYINSTALLDIR)_ and it compiled correctly. FVCOM compiles fine, but if I add the flag 30 = -DNH (I need the non-hydrostatic option), it gives the following errors:

_mod_petsc.f90:1002:63:
 1002 |   CALL KSPSetTolerances(Ksp,RTOL,PETSC_DEFAULT_DOUBLE_PRECISION,  &_
   |                                                               1
Error: Symbol ‘petsc_default_double_precision’ at (1) has no IMPLICIT type

mod_petsc.f90:892:64:
  892 |   CALL ISLocalToGlobalMappingCreate(PETSC_COMM_SELF,PSIZE_WHALO,PTMP,ISL2G,IERR); if (IERR .ne. 0) then; call PetscErrorF(IERR,576,"mod_petsc.F"); return; endif
      |                                                                1
Error: Rank mismatch in argument ‘c’ at (1) (scalar and rank-1)

mod_petsc.f90:916:48:
  916 |   CALL MatSetLocalToGlobalMapping(A,ISL2G,IERR); if (IERR .ne. 0) then; call PetscErrorF(IERR,624,"mod_petsc.F"); return; endif
      |                                                1
Error: Type mismatch in argument ‘c’ at (1); passed INTEGER(4) to INTEGER(8)

mod_petsc.f90:996:58:
  996 |   CALL KSPSetOperators(Ksp,A,A,SAME_NONZERO_PATTERN,IERR); if (IERR .ne. 0) then; call PetscErrorF(IERR,761,"mod_petsc.F"); return; endif
      |                                                          1
Error: More actual than formal arguments in procedure call at (1)

I have been looking inside mod_petsc.F, and in the call to CALL KSPSetTolerances, if it is being compiled with NH, it uses the default variables PETSC_DEFAULT_DOUBLE_PRECISION; if not, it uses PETSC_DEFAULT_REAL.

How can I compile pets with double precision? or do I need to load them from mod_pestc? Thanks

[liesvyvall]

PETSC_DEFAULT_REAL

seems like I need to add --with-precision=double when compiling petsc

[riquitorres]

Short update on Semi-implicit + waves
PETSC 3.18.5 doesn't compile with HYPRE and mpiifx. I downloaded the latest PETSC (3.21.5) and found no problems compiling as suggested by @danesnick once I changed mpif90 for mpiifx.

[liesvyvall]

IMPLICIT NONE
#include "petsc/finclude/petsc.h"
#include "petsc/finclude/petscvec.h"
#include "petsc/finclude/petscdmda.h"
#include "petsc/finclude/petscmat.h"
#include "petsc/finclude/petscksp.h"
#include "petsc/finclude/petscpc.h"
#include "petsc/finclude/petscis.h"
#include "petsc/finclude/petscao.h"
#include "petsc/finclude/petscvec.h"
#include "petsc/finclude/petscviewer.h"

Hi Ricardo i am compiling fvcom with non_hydrostatic: can you share what do you used in this lines of code for the Wave_current_interaction?

# if defined (NH)
  CALL MatCreateMPIAIJ(MPI_FVCOM_GROUP,PSIZE,PSIZE,PsizeGL,PsizeGL,0,DNNZ,0,ONNZ,A,IERR);CHKERRQ(IERR)
# endif

# if defined (SEMI_IMPLICIT) || (NH)
# if defined (PETSC_A) || (PETSC_B)  /* Siqi Li, PETSC@20230227 */
  CALL MatCreateMPIAIJ(MPI_FVCOM_GROUP,N_VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_EL,IERR);CHKERRQ(IERR)
# else
  CALL MatCreateAIJ(MPI_FVCOM_GROUP,N_VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_EL,IERR);CHKERRQ(IERR)
  CALL MatSetOption(A_EL, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE, IERR);CHKERRQ(IERR)
# endif
# endif

# if defined (WAVE_CURRENT_INTERACTION) && !defined (EXPLICIT)
  CALL MatCreateMPIAIJ(MPI_FVCOM_GROUP,N_VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_WAVE,IERR);CHKERRQ(IERR)
# endif

I have tried with MatCreateMPIAIJ(MPI_FVCOM_GROUP,N_VERTS,N_VERTS,MGL,MGL,0,DNNZ2,0,ONNZ2,A_EL,IERR);CHKERRQ(IERR) but in this case compilation gives me this error:

ld: mod_petsc.o: en la función `mod_petsc_mp_petsc_alloca_.V':
mod_petsc.f90:(.text+0xd4c6): referencia a `matcreatempiaij_' sin definir
ld: mod_petsc.f90:(.text+0xd524): referencia a `matcreatempiaij_' sin definir
ld: mod_petsc.o: en la función `mod_petsc_mp_petsc_alloca_.A':
mod_petsc.f90:(.text+0xf3dc): referencia a `matcreatempiaij_' sin definir
ld: mod_petsc.f90:(.text+0xf43a): referencia a `matcreatempiaij_' sin definir
make: *** [makefile:135: fvcom] Error 1

And if I Use

  CALL MatSetOption(A_EL, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE, IERR);CHKERRQ(IERR)

Then when running the model I got this errors:

[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: General MPI error
[0]PETSC ERROR: MPI error 135870213 Invalid communicator, error stack:
                PMPI_Comm_get_attr(350): MPI_Comm_get_attr(comm=0xfffffc19, comm_keyval=-1539309568, attribute_val=0x7fffffff9748, flag=0x7fffffff9438) failed
                MPII_Comm_get_attr(259): MPIR_Comm_get_attr(comm=0xfffffc19, comm_keyval=-1539309568, attribute_val=0x7fffffff9748, flag=0x7fffffff9438) failed
                MPII_Comm_get_attr(55).: Invalid communicator
[0]PETSC ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc!
[0]PETSC ERROR: [1]PETSC ERROR: General MPI error
[1]PETSC ERROR: MPI error 135870213 Invalid communicator, error stack:
                PMPI_Comm_get_attr(350): MPI_Comm_get_attr(comm=0xfffffc19, comm_keyval=-1539309568, attribute_val=0x7fffffff9748, flag=0x7fffffff9438) failed
                MPII_Comm_get_attr(259): MPIR_Comm_get_attr(comm=0xfffffc19, comm_keyval=-1539309568, attribute_val=0x7fffffff9748, flag=0x7fffffff9438) failed
                MPII_Comm_get_attr(55).: Invalid communicator

[1]PETSC ERROR: #1 PetscCommDuplicate()
[0]PETSC ERROR: #2 PetscHeaderCreate_Private()
[1]PETSC ERROR: #3 PetscHeaderCreate_Function()
[1]PETSC ERROR: #4 MatCreate()
[0]PETSC ERROR: [1]PETSC ERROR: #5 MatCreateAIJ()

I am using: petsc-3.21.5, FVCOM4.4.7, with
CPP = /usr/bin/cpp
COMPILER = -DIFORT
CC = mpiicc
CXX = mpicxx
CFLAGS = -O3
FC = mpiifort

[riquitorres]

Hi Liesvy,
There are a few instances in which the code needs amending. I have uploaded the file with the modifications here. If you do a meld with your version, you will see where I added the changes.
I did managed to compile without problems but I haven't managed an error free simulation yet. So I can't guarantee the changes are correct!
Regards
Ricardo
mod_petsc.txt

[liesvyvall]

Hi Liesvy, There are a few instances in which the code needs amending. I have uploaded the file with the modifications here. If you do a meld with your version, you will see where I added the changes. I did managed to compile without problems but I haven't managed an error free simulation yet. So I can't guarantee the changes are correct! Regards Ricardo mod_petsc.txt

Thanks, same here, I modified mod_petsc.F and mod_non_hydro.F, I am using petsc-2.3.3-p16 because other versions doesn't compile. Now I am compiling without error, but got some errors when running, I put off the data_assimilation flag cause it gives me more errors in compilation. When running with NH flag gives me this

DEPTH IN NODE:         1387 ; IS LESS THAN MIN_DEPTH
 ADJUST BATHYMETRY AT THIS (THESE) LOCATION(S) OR
 RECOMPILE FVCOM WITH FLOODING/DRYING FORMULATION
 STOPPING....

I have run this same input files without NH flag, when I try to compile with wet_dry gives more errors in mod_petsc.F. I have modified the depth_check.F to eliminate this error. Now I got this PETSc SOLVER HAS DIVERGED: STOPPING...

I have uploaded the files with modifications too. Thanks

mod_non_hydro.F.txt
mod_petsc-5.F.txt