User docs plus fixes

docs/user/Installation.qmd

@@ -0,0 +1,91 @@
+---
+title: "Installation Guide"
+subtitle: "Hints and Tips"
+authors:
+- name: Underworld Team
+
+exports:
+- format: html
+---
+
+## Source code build using mamba
+
+We recommend that you use a `conda` / `mamba` virtual environment to build `underworld3` whenever you want to install on a personal workstation or laptop (linux or macOS).The [`mamba` documentation](https://mamba.readthedocs.io/en/latest/index.html) will help you get started if you are not familiar with the philososphy and practice of this package management system.
+
+The `underworld3` build / run-time dependencies can be installed using
+
+```bash
+    (base) % git clone -b development --single-branch https://github.com/underworldcode/underworld3 /path/to/underworld3
+    (base) % cd /path/to/underworld3
+    (base) % mamba env create -n uw3 -f environment.yml
+    (base) % mamba activate uw3
+
+    (uw3) % pip install .
+```
+
+## Source code build using mamba and existing PETSc
+
+If you have an exisiting build of `PETSc` that you need to use, the following
+instructions work well. We create the environment as before but remove the pre-
+installed `PETSc` and `petsc4py`.
+
+```bash
+    (base) % git clone -b development --single-branch https://github.com/underworldcode/underworld3 /path/to/underworld3
+    (base) % cd /path/to/underworld3
+    (base) % mamba env create -n uw3p -f environment.yml
+    (base) % mamba activate uw3p
+
+    (uw3p) % mamba remove petsc4py
+    (uw3p) % mamba remove petsc
+```
+
+Next set up PETSc as you like it and build it using the tools
+within the current mamba virtual environment. Set the `PETSC_ARCH`
+environment variable to the name of this virtual environment to keep things
+from becoming muddled. See [PETSc Installation]() for details on
+how to configure and build what you need.
+
+```bash
+
+    (uw3p) % export PETSC_DIR=/path/to/petsc
+    (uw3p) % export PETSC_ARCH="uw3p"
+    (uw3p) % git clone -b release https://gitlab.com/petsc/petsc.git $PETSC_DIR
+    (uw3p) % cd $PETSC_DIR
+    (uw3p) % # Configure & Build step
+```
+Next we `pip install` the petsc4py that we just built into the mamba virtual environment. This
+ensures that `petsc4py` is available and also that its internal configuration points to the `PETSc` installation we just created. This way we don't need to manage environment variables to point to the build that corresponds to a given virtual environment.
+
+```bash
+    (uw3p) % mamba install cmake
+    (uw3p) % cd $PETSC_DIR/src/binding/petsc4py
+    (uw3p) % pip install .
+```
+
+Building `underworld3` against this version of `PETSc` should be identical to the mamba version.
+
+```bash
+    (uw3p) % cd /path/to/underworld3
+    (uw3p) % pip install .
+```
+
+#### Did it work ?
+
+First run the tests:
+
+```bash
+    (uw3p) % cd /path/to/underworld3
+    (uw3p) % source test.sh
+```
+
+If you have problems, contact us via the `underworl3` [GitHub issue tracker](https://github.com/underworldcode/underworld3/issues)
+
+## Docker container
+
+Windows users, if you don't want to create a linux partition, you can use our containers with `docker` or `podman`. As the code is still in active development, we are not always able to provide containers for each change to the development branch. We ask that you reach out to us on our [GitHub issue tracker](https://github.com/underworldcode/underworld3/issues) to ask for information (use the "question" label).
+
+## HPC builds
+
+`Underworld3` is inherently parallel and designed for use in high performance computing environments. The symbolic layer is relatively lightweight and should not adversely affect launch time (drawing down libraries from disk) or execution time (very few calculations are done in python itself).
+
+In the HPC environment you may find it difficult to control the software stack. You will need to ensure that you can build against PETSc 3.21 or higher, since important functionality that we need is only available from that release onwards. We are happy to provide assistance with builds on specific machines and ask that you contact us through the [GitHub issue tracker](https://github.com/underworldcode/underworld3/issues) so that other people are able to browse the issues and their fixes.

docs/user/NextSteps.qmd

@@ -12,31 +12,86 @@ exports:
 
 ### Underworld Documentation and Examples
 
+In addition to the notebooks in this brief set of examples, there are a number of sources of information on using `Underworld3` that you can access:
+
+  - [The Underworld Website / Blog](https://www.underworldcode.org)
+
+  - [The API documentation](https://underworldcode.github.io/underworld3/development_api/index.html)
+  (all the modules and functions and their full sets of arguments) is automatically generated from the source code and uses the same rich markdown content as the notebook help text.
+
+  - The [`underworld3` GitHub repository](https://github.com/underworldcode/underworld3) is the most active development community for the code.
+
 
 ### Benchmarks
 
+The [Underworld3 Benchmarks Repository](https://github.com/underworld-community/UW3-benchmarks) is a useful place to find community benchmarks coded in `underworld3` along with accuracy and convergence analysis. This is an open repository where you can make a pull request with new benchmark submissions once you get the hang of things.
 
 ### The Underworld Community
 
-Sharing codes / fixes
-
-Sharing models
+The [Underworld Community](https://github.com/underworld-community) organisation on Github is a collection of contributed repositories from the underworld user community for all versions of the code and now includes scripts for underworld3.
 
 ### Parallel Execution
 
-Advise using jupytext to render notebooks at .py files. This is helpful
-in shared environments because the files contain no output and play
-better with version control.
+`Underworld3` is inherently parallel and designed for high performance computing. The symbolic layer is a relatively thin veneer that sits on top of the `PETSc` machinery. A script that has been developed in `jupyter` should be transferrable to an HPC environment with no changes (except that inherently serial operations such as visualisation and model reduction are best left to postprocessing / co-processing).
+
+We recommend installing `jupytext` which allows either a seamless two-way conversion (or pairing) between the `ipynb` format and an annotated python script, or the ability to work entirely with annoted python scripts and not use `ipynb` at all. The python form does not store the output of notebook cells but there are advantages to this when scripts are under version control.
+
+Almost all of our notebook examples are annotated python for this reason.An exception is the collection of notebooks in this quick-start guide because we want to show you the rendered output in the static web pages.
+
+```bash
+    mpirun -np 1024 python3 Modelling_Script.py -uw_resolution 96
+```
+
+The main difference between the notebook development environment and HPC is the lack of interactivity, particularly in sending parameters to the script at launch time. Typically, we expect the HPC version to be running at much higher resolution, or for many more timesteps than the development notebook. We use the `PETSc` command line parsing machinery to generate notebooks that also can ingest run-time parameters from a script (as above).
+
+
+### Advanced capabilities
+
+Digging a bit deeper into `underworld3`, there are many capabilities that require a clear understanding of the concepts that underlie the implementation. The following examples are not *plug-and-play*, but they do only require python coding using the `underworld3` API and no detailed knowledge of `petsc4py` or `PETSc`. [Get in touch with us](https://github.com/underworldcode/underworld3/issues) if you want to try this sort of thing but can't figure it out for yourself.
 
-Not such a good idea if you want to have a tutorial with outputs (like this one)
+#### Deforming meshes
+
+In [Example 8](Notebooks/8-Particle_Swarms.ipynb), we made small variations to the mesh to conform to basal topography. We did not remesh, so we had to be careful to apply a smooth, analytic displacement to every node. For more general free-surface models, we need to calculate a smooth function using the computed boundary motions (e.g, solving a poisson equation with known boundary displacements as boundary conditions). We need to step through time and it is common to stabilize the surface motions through a surface integral term that accounts for the interface displacement during the timestep. The example below shows an `underworld3` forward model with internal loads timestepped close to isostatic equilibrium.
+
+![*Stokes flow driven by buoyancy in an annulus defined by two embedded surfaces within an enveloping disk mesh. The surfaces deform in response to the flow. The embedding medium has a very low viscosity but still acts to damp rotational modes. The outer boundary of the disk can be set to a far-field gravitational potential for whole-Earth relaxation models*](media/RelaxingMesh.png){width=50%}
+
+In a more general case, we need to account to horizontal motions. This is more complicated because the horizontal velocities can be large even when vertical equilibrium is achieved. So we need to solve for the advected component of vertical motion in addition to the local component. Hooray for symbolic algebra !
+
+#### Weak / penalty boundary conditions
+
+[Example 8](Notebooks/8-Particle_Swarms.ipynb) introduced the idea of penalty-based boundary conditions where the constraint is weakly enforced by providing a large penalty for violation of the condition. This is very flexible as the penalizing conditions can be adjusted during the run, including changing which part of the boundary is subject to constraints based on the solution or a coupled problem. The channel flow model shown below has a boundary condition that depends on a externally sourced model for ponded material at the base that is derived from a simple topography filling algorithm.
+
+```{=html}
+<center>
+<iframe src="media/pyvista/ChannelFlow.html" width="600" height="300">
+</iframe>
+</center>
+```
+*Live Image: Stokes flow in a channel with multiple obstructions. Flow is driven from the inlet (a velocity boundary condition). The geometry was contructed with `gmsh`. This is an example for education which demonstrates the emergence of an large-scale pressure gradient as a result of the presence of the obstructions, and also the dispersion of tracers through the complicated flow geometry*
+
+The penalty approach does allow the solution to deviate from the exact value of the boundary condition, in a similar way to the iterative solvers delivering a solution to within a specified tolerance. There are some cases, for example, enforcing plate motions at the surface, where there are uncertainties in the applied boundary conditions and that these uncertainties may vary in space and time.
+
+#### Mesh Adaptation
+
+It is also possible to use the PETSc mesh adaption capabilities, to refine the resolution of a model where it is most needed and coarsen elsewhere. Dynamically-adapting meshes are possible but the interface is very low level at present, particularly in parallel.
+
+```{=html}
+<center>
+<iframe src="media/pyvista/AdaptedSphere.html" width="600" height="300">
+</iframe>
+</center>
+```
+*Live Image: Static mesh adaptation to the slope of a field. The driving buoyancy term is three plume-like upwellings and the slope of this field is shown in colour (red high, blue low). The adapted mesh is shown in green.*
 
+```python
 
+    # t is the driving "temperature". We form an isotropic refinement metric from its slope
 
-<!--
-```{iframe} https://github.com/underworldcode/underworld3/raw/VEP-preliminary-implementation/Documentation/media/Sinker.html
-:width: 100%
-:align: center
+    refinement_fn = 1.0 + sympy.sqrt(
+          t.diff(x) ** 2
+        + t.diff(y) ** 2
+        + t.diff(z) ** 2
+    )
 
-Caption
+    icoord, meshA = adaptivity.mesh_adapt_meshVar(mesh0, refinement_fn, Metric, redistribute=True)
 ```
--->

docs/user/Notebooks/.jupyter/desktop-settings.json

@@ -1,3 +1,3 @@
 {
-  "pythonPath": "/Users/lmoresi/mambaforge/envs/underworld3p/bin/python3"
+  "pythonPath": "/Users/lmoresi/mambaforge/envs/underworld3/bin/python"
 }

docs/user/Notebooks/1-Meshes.ipynb

@@ -64,7 +64,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 9,
    "id": "5a4437ff-63ae-4ffa-9f2b-8ec1b8b4baeb",
    "metadata": {
     "editable": true,
@@ -74,15 +74,7 @@
     },
     "tags": []
    },
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[esdhcp-137.anu.edu.au:81197] shmem: mmap: an error occurred while determining whether or not /var/folders/tx/95gr762j29z4tt5d1dnqlgth0000gn/T//ompi.esdhcp-137.501/jf.0/1727463424/sm_segment.esdhcp-137.501.66f70000.0 could be created.\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "#| output: false # Suppress warnings in html version\n",
     "\n",
@@ -274,7 +266,7 @@
        "        "
       ],
       "text/plain": [
-       "<IPython.lib.display.IFrame at 0x17c2f2c10>"
+       "<IPython.lib.display.IFrame at 0x155c7fc90>"
       ]
      },
      "execution_count": 6,
@@ -476,7 +468,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.7"
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,

docs/user/Notebooks/2-Variables.ipynb

@@ -367,7 +367,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.7"
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,

docs/user/Notebooks/3-Symbolic_Forms.ipynb

@@ -37,7 +37,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 23,
    "id": "5a4437ff-63ae-4ffa-9f2b-8ec1b8b4baeb",
    "metadata": {
     "editable": true,
@@ -46,15 +46,7 @@
     },
     "tags": []
    },
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[Lyrebird.local:19272] shmem: mmap: an error occurred while determining whether or not /var/folders/tx/95gr762j29z4tt5d1dnqlgth0000gn/T//ompi.Lyrebird.501/jf.0/95551488/sm_segment.Lyrebird.501.5b20000.0 could be created.\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "import underworld3 as uw\n",
     "import numpy as np\n",
@@ -329,7 +321,31 @@
     {
      "data": {
       "text/markdown": [
-       "${ \\nabla \\times \\left(\\cal{S} \\mathbf{v}\\right) \\hspace{ 0.0pt } }$"
+       "**Class**: <class 'underworld3.function.expressions.UWexpression'>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.Markdown object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/latex": [
+       "$\\quad$$\\displaystyle { \\nabla \\times \\left(\\cal{S} \\mathbf{v}\\right) \\hspace{ 0.0pt } }$$=$$\\displaystyle - {\\cal{S}}(\\mathbf{x}) {\\mathbf{v}}_{ 0,1}(\\mathbf{x}) + {\\cal{S}}(\\mathbf{x}) {\\mathbf{v}}_{ 1,0}(\\mathbf{x}) + {\\cal{S}}_{,0}(\\mathbf{x}) {\\mathbf{v}}_{ 1 }(\\mathbf{x}) - {\\cal{S}}_{,1}(\\mathbf{x}) {\\mathbf{v}}_{ 0 }(\\mathbf{x})$"
+      ],
+      "text/plain": [
+       "<IPython.core.display.Latex object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/markdown": [
+       "$\\quad$**Description:**  Curl of {\\mathbf{v}}"
       ],
       "text/plain": [
        "<IPython.core.display.Markdown object>"
@@ -368,10 +384,10 @@
     "curl_sv = uw.function.expression(\n",
     "                    r\"\\nabla \\times \\left(\\cal{S} \\mathbf{v}\\right)\",\n",
     "                    mesh.vector.curl(s.sym * v.sym),\n",
-    "                    f\"Curl of {v.symbol}\"\n",
+    "                    rf\"Curl of {v.symbol}\"\n",
     "                ) \n",
     "\n",
-    "display(curl_sv)\n",
+    "curl_sv.view()\n",
     "display(curl_sv.sym) \n",
     "\n",
     "curl_sv + 1"
@@ -603,7 +619,7 @@
        "        "
       ],
       "text/plain": [
-       "<IPython.lib.display.IFrame at 0x302cfdc90>"
+       "<IPython.lib.display.IFrame at 0x152a4eb10>"
       ]
      },
      "execution_count": 15,
@@ -810,7 +826,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 30,
+   "execution_count": null,
    "id": "59a1273d-ee09-479a-9788-d2e6b399e9b8",
    "metadata": {},
    "outputs": [],
@@ -833,7 +849,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.7"
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,