Skip to content

Commit f31b552

Browse files
MdHassanPappuMdHassanPappu
committed
rewrite Fenicsx build instruction with Spack
1 parent 5f0a0c4 commit f31b552

File tree

1 file changed

+177
-136
lines changed

1 file changed

+177
-136
lines changed

‎docs/software/cae/fenics.md‎

Lines changed: 177 additions & 136 deletions
Original file line numberDiff line numberDiff line change
@@ -1,161 +1,202 @@
11
[![](https://fenicsproject.org/pub/tutorial/sphinx1/_static/fenics_banner.png){: style="width:200px;float: right;" }](https://fenicsproject.org/)
2-
[FEniCS](https://fenicsproject.org/) is a popular open-source (LGPLv3) computing platform for
3-
solving partial differential equations (PDEs).
4-
FEniCS enables users to quickly translate scientific models
5-
into efficient finite element code. With the high-level
6-
Python and C++ interfaces to FEniCS, it is easy to get started,
7-
but FEniCS offers also powerful capabilities for more
8-
experienced programmers. FEniCS runs on a multitude of
9-
platforms ranging from laptops to high-performance clusters.
10-
11-
## How to access the FEniCS through [Anaconda](https://www.anaconda.com/products/individual)
12-
The following steps provides information about how to installed
13-
on your local path.
14-
```bash
15-
# From your local computer
16-
$ ssh -X iris-cluster # OR ssh -Y iris-cluster on Mac
172

18-
# Reserve the node for interactive computation with grahics view (plots)
19-
$ si --x11 --ntasks-per-node 1 -c 4
20-
# salloc -p interactive --qos debug -C batch --x11 --ntasks-per-node 1 -c 4
3+
<!-- Intro start -->
214

22-
# Go to scratch directory
23-
$ cds
5+
[FEniCS](https://fenicsproject.org/) is a popular open-source computing platform for solving partial differential equations (PDEs) using the finite element method ([FEM](https://en.wikipedia.org/wiki/Finite_element_method)). Originally developed in 2003, the earlier version is now known as legacy FEniCS. In 2020, the next-generation framework [FEniCSx](https://docs.fenicsproject.org/) was introduced, with the latest stable [release v0.9.0](https://fenicsproject.org/blog/v0.9.0/) in October 2024. Though it builds on the legacy FEniCS but introduces significant improvements over the legacy libraries. FEniCSx is comprised of the libraries [UFL](https://github.com/FEniCS/ufl), [Basix](https://github.com/FEniCS/basix), [FFCx](https://github.com/FEniCS/ffcx), and [DOLFINx](https://github.com/FEniCS/dolfinx) which are the build blocks of it. And new users are encouraged to adopt [FEniCSx](https://docs.fenicsproject.org/) for its modern features and active development support.
246

25-
/scratch/users/<login> $ Anaconda3-2020.07-Linux-x86_64.sh
26-
/scratch/users/<login> $ chmod +x Anaconda3-2020.07-Linux-x86_64.sh
27-
/scratch/users/<login> $ ./Anaconda3-2020.07-Linux-x86_64.sh
287

29-
Do you accept the license terms? [yes|no]
30-
yes
31-
Anaconda3 will now be installed into this location:
32-
/home/users/<login>/anaconda3
8+
<!-- // Tutorials: https://jsdokken.com/dolfinx-tutorial/index.html -->
339

34-
- Press ENTER to confirm the location
35-
- Press CTRL-C to abort the installation
36-
- Or specify a different location below
3710

38-
# You can choose your path where you want to install it
39-
[/home/users/<login>/anaconda3] >>> /scratch/users/<login>/Anaconda3
11+
<!-- Intro end -->
4012

41-
# To activate the anaconda
42-
/scratch/users/<login> $ source /scratch/users/<login>/Anaconda3/bin/activate
13+
## Installation of FEniCSx
4314

44-
# Install the fenics in anaconda environment
45-
/scratch/users/<login> $ conda create -n fenicsproject -c conda-forge fenics
15+
FEniCSx can be installed on [ULHPC](https://www.uni.lu/research-en/core-facilities/hpc/) systems using [Easybuild](https://docs.easybuild.io) or [Spack](https://spack.io/), Below are detailed instructions for each method,
4616

47-
# Install matplotlib for the visualization
48-
/scratch/users/<login> $ conda install -c conda-forge matplotlib
49-
```
50-
Once you have installed the anaconda, you can always
51-
activate it by calling the `source activate` path where `anaconda`
52-
has been installed.
5317

54-
## Working example
55-
### Interactive mode
56-
```bash
57-
# From your local computer
58-
$ ssh -X iris-cluster # or ssh -Y iris-cluster on Mac
5918

60-
# Reserve the node for interactive computation with grahics view (plots)
61-
$ si --ntasks-per-node 1 -c 4 --x11
62-
# salloc -p interactive --qos debug -C batch --x11 --ntasks-per-node 1 -c 4
19+
### Building FEniCS With Spack
6320

64-
# Activate anaconda
65-
$ source /${SCRATCH}/Anaconda3/bin/activate
6621

67-
# activate the fenicsproject
68-
$ conda activate fenicsproject
22+
Building FEniCSx with Spack requires that Spack is already installed, configured, and its environment sourced on the [ULHPC] system. If Spack is not yet configured, follow the [spack documentation](../../environment/spack.md) for installation and configuration.
6923

70-
# execute the Poisson.py example (you can uncomment the plot lines in Poission.py example)
71-
$ python3 Poisson.py
72-
```
24+
!!! note
25+
Spack can a good choice to build FEniCSx with its many complex dependencies, leveraging the system-provided packages defined in ~/.spack/packages.yaml for optimal performance.
26+
27+
Create and Activate a Spack Environment:
28+
29+
To maintain an isolated installation, create a dedicated Spack environment in a chosen directory.
30+
The following example builds FEniCSx in the `home` directory:
31+
32+
cd ~
33+
spack env create -d fenicsx-main-20230126/
34+
spack env activate fenicsx-main-20230126/
35+
36+
37+
Add the core FEniCSx components and common dependencies:
38+
39+
spack add py-fenics-dolfinx@0.9.0+petsc4py fenics-dolfinx+adios2+petsc adios2+python petsc+mumps
40+
41+
# Change @0.9.0 to any version in the above if you want a another version.
42+
spack concretize
43+
spack install -j16
44+
45+
46+
!!! note
47+
48+
`spack concretize` resolves all dependencies and selects compatible versions for the specified packages. `-j16` sets the number of parallel build jobs. Using a higher number can speed up the build but should be chosen based on available CPU cores and cluster policies.
49+
50+
51+
52+
or the same directly in `spack.yaml` in `$SPACK_ENV`
53+
54+
spack:
55+
# add package specs to the `specs` list
56+
specs:
57+
- py-fenics-dolfinx@0.9.0+petsc4py
58+
- fenics-dolfinx+adios2+petsc
59+
- petsc+mumps
60+
- adios2+python
61+
62+
view: true
63+
concretizer:
64+
unify: true
65+
66+
The following are also commonly used in FEniCS scripts and may be useful
67+
68+
spack add gmsh+opencascade py-numba py-scipy py-matplotlib
69+
70+
It is possible to build a specific version (git ref) of DOLFINx.
71+
Note that the hash must be the full hash.
72+
It is best to specify appropriate git refs on all components.
73+
74+
# This is a Spack Environment file.
75+
#
76+
# It describes a set of packages to be installed, along with
77+
# configuration settings.
78+
spack:
79+
# add package specs to the `specs` list
80+
specs:
81+
- fenics-dolfinx@git.4f575964c70efd02dca92f2cf10c125071b17e4d=main+adios2
82+
- py-fenics-dolfinx@git.4f575964c70efd02dca92f2cf10c125071b17e4d=main
83+
84+
- py-fenics-basix@git.2e2a7048ea5f4255c22af18af3b828036f1c8b50=main
85+
- fenics-basix@git.2e2a7048ea5f4255c22af18af3b828036f1c8b50=main
86+
87+
- py-fenics-ufl@git.b15d8d3fdfea5ad6fe78531ec4ce6059cafeaa89=main
88+
89+
- py-fenics-ffcx@git.7bc8be738997e7ce68ef0f406eab63c00d467092=main
90+
91+
- fenics-ufcx@git.7bc8be738997e7ce68ef0f406eab63c00d467092=main
92+
93+
- petsc+mumps
94+
- adios2+python
95+
view: true
96+
concretizer:
97+
unify: true
98+
99+
It is also possible to build only the C++ layer using
100+
101+
spack add fenics-dolfinx@main+adios2 py-fenics-ffcx@main petsc+mumps
102+
103+
To rebuild FEniCSx from main branches inside an existing environment
104+
105+
spack install --overwrite -j16 fenics-basix py-fenics-basix py-fenics-ffcx fenics-ufcx py-fenics-ufl fenics-dolfinx py-fenics-dolfinx
106+
107+
#### Testing the build
108+
109+
Quickly test the build with
110+
111+
srun python -c "from mpi4py import MPI; import dolfinx"
112+
113+
#### Using the build
114+
115+
See the uni.lu documentation for full details - using the environment should be as
116+
simple as adding the following where `...` is the name/folder of your environment.
117+
118+
#!/bin/bash -l
119+
source $HOME/spack/share/spack/setup-env.sh
120+
spack env activate ...
121+
122+
#### Known issues
123+
124+
Workaround for inability to find gmsh Python package:
125+
126+
export PYTHONPATH=$SPACK_ENV/.spack-env/view/lib64/:$PYTHONPATH
127+
128+
Workaround for inability to find adios2 Python package:
129+
130+
export PYTHONPATH=$(find $SPACK_ENV/.spack-env -type d -name 'site-packages' | grep venv):$PYTHONPATH
131+
132+
133+
### Building FEniCS With EasyBuild
73134

74-
### Batch script
75-
```bash
76-
#!/bin/bash -l
77-
#SBATCH -J FEniCS
78-
#SBATCH -N 1
79-
###SBATCH -A <project name>
80-
###SBATCH --ntasks-per-node=1
81-
#SBATCH -c 1
82-
#SBATCH --time=00:05:00
83-
#SBATCH -p batch
84-
85-
echo "== Starting run at $(date)"
86-
echo "== Job ID: ${SLURM_JOBID}"
87-
echo "== Node list: ${SLURM_NODELIST}"
88-
echo "== Submit dir. : ${SLURM_SUBMIT_DIR}"
89-
90-
# activate the anaconda source
91-
source ${SCRATCH}/Anaconda3/bin/activate
92-
93-
# activate the fenicsproject from anaconda
94-
conda activate fenicsproject
95-
96-
# execute the poisson.py through python
97-
srun python3 Poisson.py
98-
```
99135

100136
### Example (Poisson.py)
101137
```bash
102-
# FEniCS tutorial demo program: Poisson equation with Dirichlet conditions.
103-
# Test problem is chosen to give an exact solution at all nodes of the mesh.
104-
# -Laplace(u) = f in the unit square
105-
# u = u_D on the boundary
106-
# u_D = 1 + x^2 + 2y^2
107-
# f = -6
108-
109-
from __future__ import print_function
110-
from fenics import *
111-
import matplotlib.pyplot as plt
112-
113-
# Create mesh and define function space
114-
mesh = UnitSquareMesh(8, 8)
115-
V = FunctionSpace(mesh, 'P', 1)
116-
117-
# Define boundary condition
118-
u_D = Expression('1 + x[0]*x[0] + 2*x[1]*x[1]', degree=2)
119-
120-
def boundary(x, on_boundary):
121-
return on_boundary
122-
123-
bc = DirichletBC(V, u_D, boundary)
124-
125-
# Define variational problem
126-
u = TrialFunction(V)
127-
v = TestFunction(V)
128-
f = Constant(-6.0)
129-
a = dot(grad(u), grad(v))*dx
130-
L = f*v*dx
131-
132-
# Compute solution
133-
u = Function(V)
134-
solve(a == L, u, bc)
135-
136-
# Plot solution and mesh
137-
#plot(u)
138-
#plot(mesh)
139-
140-
# Save solution to file in VTK format
141-
vtkfile = File('poisson/solution.pvd')
142-
vtkfile << u
143-
144-
# Compute error in L2 norm
145-
error_L2 = errornorm(u_D, u, 'L2')
146-
147-
# Compute maximum error at vertices
148-
vertex_values_u_D = u_D.compute_vertex_values(mesh)
149-
vertex_values_u = u.compute_vertex_values(mesh)
138+
139+
# Demo possion problem
140+
# https://docs.fenicsproject.org/dolfinx/main/python/demos/demo_poisson.html
141+
142+
from mpi4py import MPI
143+
from petsc4py.PETSc import ScalarType
144+
150145
import numpy as np
151-
error_max = np.max(np.abs(vertex_values_u_D - vertex_values_u))
152146

153-
# Print errors
154-
print('error_L2 =', error_L2)
155-
print('error_max =', error_max)
147+
import ufl
148+
from dolfinx import fem, mesh
149+
from dolfinx.fem.petsc import LinearProblem
150+
151+
# Create mesh
152+
msh = mesh.create_rectangle(
153+
comm=MPI.COMM_WORLD,
154+
points=((0.0, 0.0), (2.0, 1.0)),
155+
n=(32, 16),
156+
cell_type=mesh.CellType.triangle,
157+
)
158+
159+
# Function space
160+
V = fem.functionspace(msh, ("Lagrange", 1))
161+
162+
# Boundary facets (x=0 and x=2)
163+
facets = mesh.locate_entities_boundary(
164+
msh,
165+
dim=(msh.topology.dim - 1),
166+
marker=lambda x: np.isclose(x[0], 0.0) | np.isclose(x[0], 2.0),
167+
)
168+
dofs = fem.locate_dofs_topological(V=V, entity_dim=1, entities=facets)
169+
170+
# Dirichlet BC u = 0
171+
bc = fem.dirichletbc(value=ScalarType(0), dofs=dofs, V=V)
172+
173+
# Variational problem
174+
u = ufl.TrialFunction(V)
175+
v = ufl.TestFunction(V)
176+
x = ufl.SpatialCoordinate(msh)
177+
f = 10 * ufl.exp(-((x[0] - 0.5) ** 2 + (x[1] - 0.5) ** 2) / 0.02)
178+
g = ufl.sin(5 * x[0])
179+
a = ufl.inner(ufl.grad(u), ufl.grad(v)) * ufl.dx
180+
L = ufl.inner(f, v) * ufl.dx + ufl.inner(g, v) * ufl.ds
181+
182+
# Create problem (no petsc_options_prefix in 0.9.0)
183+
problem = LinearProblem(
184+
a,
185+
L,
186+
bcs=[bc],
187+
petsc_options={"ksp_type": "preonly", "pc_type": "lu", "ksp_error_if_not_converged": True},
188+
)
189+
190+
# Solve
191+
uh = problem.solve()
192+
193+
# Only print from rank 0 to avoid MPI spam
194+
if MPI.COMM_WORLD.rank == 0:
195+
print("First 10 values of the solution vector:", uh.x.array[:10])
196+
197+
assert isinstance(uh, fem.Function)
198+
156199

157-
# Hold plot
158-
#plt.show()
159200
```
160201
161202
## Additional information

0 commit comments

Comments
 (0)