Error when building moose on the INL HPC #26011

aaelmeli · 2023-11-09T18:52:43Z

aaelmeli
Nov 9, 2023

Hi

Recently I was no longer able to run my application on INL HPC, I think they updated some compilers or so. I attempt to rebuild moose again but got errors. I tried to update petsc, libmesh through running ./update_and_rebuild_ptsc.sh and ./update_and_rebuild_libmsh.sh
still I can not build moose. Please see the error below: (it seems was almost there, it linked all the libraries, etc, but then failed.

/home/elmeabde/sawtooth1/projects/moose/framework/libmoose-opt.so: undefined reference to `hit::Section::render(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int)'
collect2: error: ld returned 1 exit status
make: *** [/home/elmeabde/sawtooth1/projects/moose/framework/app.mk:442: /home/elmeabde/sawtooth1/projects/moose/test/moose_test-opt] Error 1

I tried again to update mamba, but I am still getting the following error.
Any idea?

error    libmamba Could not open lockfile '/apps/moose/stack/moose-tools-2023.10.19/pkgs/cache/cache.lock'
warning  libmamba Cache file "/home/elmeabde/.conda/pkgs/cache/497deca9.json" was modified by another program
error    libmamba Could not open lockfile '/apps/moose/stack/moose-tools-2023.10.19/pkgs/cache/cache.lock'
warning  libmamba Cache file "/home/elmeabde/.conda/pkgs/cache/09cdf8bf.json" was modified by another program

Answered by milljm

Nov 9, 2023

I think you need to clean your MOOSE repo, and then try to build again:

cd /home/elmeabde/sawtooth1/projects/moose
# The following commands will delete any untracked files/changes.
git clean -xfd; git submodule foreach --recursive git clean -xfd

And then attempt to rebuild.

As for your attempt at updating Mamba, you cannot. You are using a Conda instance provided to you by Sawtooth (not owned by you). That Conda instance is up to date as well (I maintain it).

View full answer

milljm · 2023-11-09T19:44:05Z

milljm
Nov 9, 2023
Maintainer

I think you need to clean your MOOSE repo, and then try to build again:

cd /home/elmeabde/sawtooth1/projects/moose
# The following commands will delete any untracked files/changes.
git clean -xfd; git submodule foreach --recursive git clean -xfd

And then attempt to rebuild.

As for your attempt at updating Mamba, you cannot. You are using a Conda instance provided to you by Sawtooth (not owned by you). That Conda instance is up to date as well (I maintain it).

10 replies

aaelmeli Nov 10, 2023
Author

These are the modules loaded to my environment.

  1) use.moose                     6) zlib/1.2.13-gcc-12.3.0-c2ww   11) zstd/1.4.4-gcc-10.5.0-yue3     16) moose-tools/2023-10-19
  2) flex/2.6.4-gcc-12.3.0-cnsv    7) cmake/3.27.7-gcc-12.3.0-5cfk  12) gcc/12.3.0-gcc-10.5.0-vx2f     17) moose-dev/2023-10-23
  3) bison/3.0.4-gcc-12.3.0-547w   8) gmp/6.1.2-gcc-10.5.0-wep3     13) mvapich2/2.3.7-1-gcc-12.3.0
  4) curl/7.61.1-gcc-12.3.0-5t73   9) mpfr/4.2.0-gcc-10.5.0-cec4    14) matlab/R2023b-gcc-12.3.0-buub
  5) ncurses/6.1-gcc-12.3.0-7zps  10) mpc/1.3.1-gcc-10.5.0-mpcb     15) patchelf/0.18.0

and this is the job.sh file

#PBS -N mooseMatlabOptimization
#PBS -l select=7:ncpus=48:mpiprocs=48
#PBS -l place=scatter:excl
#PBS -l walltime=15:0:00
#PBS -j oe
#PBS -P edu_res

cd $PBS_O_WORKDIR
module purge
source /etc/profile.d/modules.sh
module load moose-dev/2023-10-23
module load matlab
matlab < /home/elmeabde/sawtooth1/projects/matlabMooseWrapper/matlab_and_moose/inverse_solve.m  > optOutput.txt

milljm Nov 10, 2023
Maintainer

You're going to have to help me out with matlab. How can I create my own inverse_solve.m ? So far, everything has been working for me... here are my steps:

ssh sawtooth1.hpc.inl.gov
module purge
module load use.moose moose-dev matlab
module list

$ module list

Currently Loaded Modules:
  1) use.moose                     10) mpfr/4.2.0-gcc-10.5.0-cec4
  2) flex/2.6.4-gcc-12.3.0-cnsv    11) mpc/1.3.1-gcc-10.5.0-mpcb
  3) bison/3.0.4-gcc-12.3.0-547w   12) zstd/1.4.4-gcc-10.5.0-yue3
  4) curl/7.61.1-gcc-12.3.0-5t73   13) gcc/12.3.0-gcc-10.5.0-vx2f
  5) ncurses/6.1-gcc-12.3.0-7zps   14) mvapich2/2.3.7-1-gcc-12.3.0
  6) zlib/1.2.13-gcc-12.3.0-c2ww   15) moose-tools/2023-10-19
  7) cmake/3.27.7-gcc-12.3.0-5cfk  16) moose-dev/2023-10-23
  8) patchelf/0.18.0               17) matlab/R2023b-gcc-12.3.0-buub
  9) gmp/6.1.2-gcc-10.5.0-wep3

I started with a fresh clone. Ran the update scripts:

git clone https://github.com/idaholab/moose
cd moose
git checkout master
cd scripts
export METHODS=opt METHOD=opt MOOSE_JOBS=6
./update_and_rebuild_petsc.sh
./update_and_rebuild_libmesh.sh
./update_and_rebuild_wasp.sh

Then built and tested it successfully:

cd ../test
make -j 6
./run_tests -j 6

EDIT: Correction, I have a CSVDIFF failure... but that can be ignored in this case.

aaelmeli Nov 10, 2023
Author

So in the same moose/test directory, you can create a matlab script (eg. runmoose.m), in this matlab file you just need to call the run_tests file. please see how I used to run moose executables from a matlab script.

runcmd = [' cd  /home/elmeabde/sawtooth1/projects/first_test/' directoryname ' && ./first_test-opt  -i  ' num2str(filename) ' > NULL '];
    [status, cmderrmsg] = unix(runcmd);

    if (status ~= 0)
        error(['Run bash cmd error: ' num2str(status) ' ' cmderrmsg]);
    end

so in your case, the runcmd array would be

runcmd = ['./run_tests > NULL '];

milljm Nov 13, 2023
Maintainer

I am getting an entirely different error...

$ matlab < /home/milljm/projects/moose/test/runmoose.m 
MATLAB is selecting SOFTWARE OPENGL rendering.

                            < M A T L A B (R) >
                  Copyright 1984-2023 The MathWorks, Inc.
             R2023b Update 3 (23.2.0.2409890) 64-bit (glnxa64)
                              October 4, 2023

 
To get started, type doc.
For product information, visit www.mathworks.com.
 
>> >> >> >> >>>>Run bash cmd error: 1 ./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/test/lib/libmoose_test-opt.so.0)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/framework/libmoose-opt.so.0)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `CXXABI_1.3.13' not found (required by
/home/milljm/projects/moose/framework/libmoose-opt.so.0)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/framework/contrib/hit/libhit-opt.so.0)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/framework/contrib/wasp/install/lib/libwasplsp.so.04)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/framework/contrib/wasp/install/lib/libwaspcore.so.04)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/framework/contrib/wasp/install/lib/libwasphalite.so.04)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/scripts/../libmesh/installed/lib/libmesh_opt.so.0)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `CXXABI_1.3.13' not found (required by
/home/milljm/projects/moose/scripts/../libmesh/installed/lib/libmesh_opt.so.0)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/scripts/../libmesh/installed/lib/libtimpi_opt.so.11)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/petsc/arch-moose/lib/libpetsc.so.3.20)
./moose_test-opt:
/apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.29' not found (required by
/home/milljm/projects/moose/petsc/arch-moose/lib/libstrumpack.so.7.1)

But moose_test-opt -h seems to be okay:

>> [~/projects/moose/test]
$ ./moose_test-opt -h
Usage: ./moose_test-opt [<options>]

Options:
  -w --allow-unused                                 Warn about unused input file options instead of erroring.
  --app <AppName>                                   Specify the application that should be used to run the input file. This must match an application name registered to the application factory. Note that this option is case-sensitive.

<trimmed>

It looks like some sort of incompatibility with matlab and the moose-dev modules that gets loaded...

milljm Nov 13, 2023
Maintainer

I just noticed this amongst your posts:

export LD_LIBRARY_PATH = /apps/spack/opt/gcc-12.3.0/matlab-R2023b-buubiju7yunxvibh5px6d6pkmg3az2wv/lib64/:/home/elmeabde/sawtooth1/projects/moose/petsc/arch-moose/lib/:$LD_LIBRARY_PATH

(applying this, with the path to my arch-moose)... I still get the same GLIBC errors

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error when building moose on the INL HPC #26011

{{title}}

Replies: 1 comment 10 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Error when building moose on the INL HPC #26011

aaelmeli Nov 9, 2023

Replies: 1 comment · 10 replies

milljm Nov 9, 2023 Maintainer

aaelmeli Nov 10, 2023 Author

milljm Nov 10, 2023 Maintainer

aaelmeli Nov 10, 2023 Author

milljm Nov 13, 2023 Maintainer

milljm Nov 13, 2023 Maintainer

aaelmeli
Nov 9, 2023

Replies: 1 comment 10 replies

milljm
Nov 9, 2023
Maintainer

aaelmeli Nov 10, 2023
Author

milljm Nov 10, 2023
Maintainer

aaelmeli Nov 10, 2023
Author

milljm Nov 13, 2023
Maintainer

milljm Nov 13, 2023
Maintainer