GPUs and/or CUDA

[aladshaw3]

I know that PETSc has some methods to utilize GPUs and was wondering if MOOSE makes use of GPUs?

If so, how would users invoke GPUs for scaling on hybrid computer systems?

And if not, are there any future plans for making MOOSE scalable on hybrid systems (CPUs and GPUs)?

Thanks.

[GiudGiud]

Hello

There are plans. Though not sure if we have something laid out somewhere to share.
@fdkong can call Petsc for MOOSE on GPU.

Guillaume

[fdkong]

I know that PETSc has some methods to utilize GPUs and was wondering if MOOSE makes use of GPUs?

MOOSE can now seamlessly use GPUs for algebra operations (linear solvers, preconditioners) via PETSc. It will significantly speed up the simulation if it is expensive on the solvers, which is valid for many applications.

We form matrix and vector on CPU and move that to GPU to solve

In the future, we plan to explore the approaches to move more calculations on GPU. It will also depend on the resources.

[ljeft]

If my graphics card is AMD and I find that PETSC fails to fully support HIP (Mat), does this mean MOOSE fails to support HIP.

[lindsayad]

It's probably best if your mpi is GPU aware. You can configure mpich with --with-hip=yes (AMD) or --with-cuda=yes (nvidia). Similarly you need to configure petsc with --with-hip=1 or --with-cuda=1. There are a few solvers that have full GPU support. These include PETSc's native GAMG as well as hypre's boomeramg. When you've configured your mpi and petsc for cuda/hip support and built moose on top of these, then you can run with -mat_type aijcusparse -vec_type cuda or with -mat_type aijhipsparse -vec_type hip with solver types like those I just mentioned. Note that you cannot use these GPU mat types for the preconditioning matrix and use matrix-free finite differencing for the Jacobian, so you cannot use PJFNK, you must use NEWTON.

[DanoLV]

Hello. Can you tell me exactly how to build PETSc with nvcc?
I have been trying to build MOOSE with CUDA aware OpenMPI an PETSc, but my efforts have been unsuccessful until now.
OpenMPI is CUDa aware in my build, but when I try to build PETSc it fails with SUPERLU_DIST.

Uploading configure.log…

[milljm]

@DanoLV Please create a new discussion thread. You may link this discussion in your new discussion thread if you wish. But it is poor practice to ask for help for your issue, when another is also asking for help in the same thread.

[milljm]

And this goes for everyone; This thread has already been answered. It cannot have another. If this thread does not answer your question, please create a new question using a new discussion, so that it may have a new answer.