From d26092d5c6459fb83f0ed88e8d92a518b1e27e1c Mon Sep 17 00:00:00 2001 From: Erik van Sebille Date: Fri, 3 May 2024 08:45:30 +0200 Subject: [PATCH 1/3] Adding section in parcels structure tutorial on optimisation and parallelisation Following the finding in #1554 that it was not clear that dask-chunking is by default None in parcels --- .../examples/tutorial_parcels_structure.ipynb | 52 ++++++++++++++++++- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/docs/examples/tutorial_parcels_structure.ipynb b/docs/examples/tutorial_parcels_structure.ipynb index 1b98374be..d42d1b920 100644 --- a/docs/examples/tutorial_parcels_structure.ipynb +++ b/docs/examples/tutorial_parcels_structure.ipynb @@ -27,6 +27,7 @@ "2. [**ParticleSet**](#2.-ParticleSet). Define the type of particles. Also additional `Variables` can be added to the particles (e.g. temperature, to keep track of the temperature that particles experience).\n", "3. [**Kernels**](#3.-Kernels). Define and compile kernels. Kernels perform some specific operation on the particles every time step (e.g. interpolate the temperature from the temperature field to the particle location).\n", "4. [**Execution and output**](#4.-Execution-and-Output). Execute the simulation and write and store the output in a NetCDF file.\n", + "5. [**Optimising and parallelising**](#5.-Optimising-and-parallelising). Optimise and parallelise the code to run faster.\n", "\n", "We discuss each component in more detail below.\n", "\n", @@ -393,8 +394,6 @@ "source": [ "
\n", "\n", - "### A note on output chunking\n", - "\n", "Note the use of the `chunks` argument in the `pset.ParticleFile()` above. This controls the 'chunking' of the output file, which is a way to optimize the writing of the output file. The default chunking for the output in Parcels is `(number of particles in initial particleset, 1)`. \n", "Note that this default may not be very efficient if \n", "1. you use `repeatdt` to release a relatively small number of particles _many_ times during the simulation and/or\n", @@ -426,6 +425,55 @@ "\n", "- [How the output is structured and how to start your own analysis](https://docs.oceanparcels.org/en/latest/examples/tutorial_output.html)\n" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 5. Optimising and parallelising" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "On linux and macOS, Parcels can be run in parallel using MPI. This can be done by running the script with\n", + "```shell\n", + "mpirun -np python .py\n", + "``` \n", + "\n", + "The script will then run in parallel on the number of cores specified. Note that this is a fairly 'simple' implementation of paralelisation, where the number of particles is simply spread over the cores (using a `kdtree` for some optimisation). This means that the more cores you use, the less particles each core will have to handle, and the faster the simulation will run. However, the speedup is not linear, as each core will need to load its own `FieldSet`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### For more tutorials on MPI and parallelisation:\n", + "\n", + "- [Optimising the partitioning of the particles with a user-defined `partition_function`](https://docs.oceanparcels.org/en/latest/examples/documentation_MPI.html#Optimising-the-partitioning-of-the-particles-with-a-user-defined-partition_function)\n", + "- [Future developments: load balancing](https://docs.oceanparcels.org/en/latest/examples/documentation_MPI.html#Future-developments:-load-balancing)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Another good way to optimise Parcels and speed-up execution is to chunk the `FieldSet` with `dask`, using the `chunksize` argument in the `FieldSet` creation. This will allow Parcels to load the `FieldSet` in chunks. \n", + "\n", + "Using chunking can be especially useful when working with large datasets _and_ when the particles only occupy a small region of the domain.\n", + "\n", + "Note that the **default** is `chunksize=None`, which means that the `FieldSet` is loaded in its entirety. This is generally the most efficient way to load the `FieldSet` when the particles are spread out over the entire domain.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### For more tutorials chunking and dask:\n", + "\n", + "- [Chunking the FieldSet with dask](https://docs.oceanparcels.org/en/latest/examples/documentation_MPI.html#Chunking-the-FieldSet-with-dask)" + ] } ], "metadata": { From f167314743073fe235fa6565ee2fbd40dee42e39 Mon Sep 17 00:00:00 2001 From: Erik van Sebille Date: Fri, 3 May 2024 09:40:11 +0200 Subject: [PATCH 2/3] Updating link to Grawe et al (2012) because doi somehow is broken(?) --- docs/examples/tutorial_diffusion.ipynb | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/examples/tutorial_diffusion.ipynb b/docs/examples/tutorial_diffusion.ipynb index 33f469f4e..021336479 100644 --- a/docs/examples/tutorial_diffusion.ipynb +++ b/docs/examples/tutorial_diffusion.ipynb @@ -74,7 +74,7 @@ "\n", "The extra term in the M1 scheme provides extra accuracy at negligible computational cost.\n", "\n", - "The spatial derivatives in the EM and M1 schemes can be approximated by a central difference. Higher order numerical schemes (see [Gräwe et al., 2012](https://doi.org/10.1007/s10236-012-0523-y)) include higher order derivatives. Since Parcels uses bilinear interpolation, these higher order derivatives cannot be computed, meaning that higher order numerical schemes cannot be used.\n", + "The spatial derivatives in the EM and M1 schemes can be approximated by a central difference. Higher order numerical schemes (see [Gräwe et al., 2012](https://link.springer.com/article/10.1007/s10236-012-0523-y)) include higher order derivatives. Since Parcels uses bilinear interpolation, these higher order derivatives cannot be computed, meaning that higher order numerical schemes cannot be used.\n", "\n", "An overview of numerical approximations for SDEs in a particle tracking setting can be found in [Gräwe (2011)](https://doi.org/10.1016/j.ocemod.2010.10.002).\n", "\n", @@ -646,7 +646,7 @@ "\n", "Gräwe, U. (2011). “Implementation of high-order particle-tracking schemes in a water column model.” _Ocean Modelling_, 36(1), 80–89. https://doi.org/10.1016/j.ocemod.2010.10.002\n", "\n", - "Gräwe, Deleersnijder, Shah & Heemink (2012). “Why the Euler scheme in particle tracking is not enough: The shallow-sea pycnocline test case.” _Ocean Dynamics_, 62(4), 501–514. https://doi.org/10.1007/s10236-012-0523-y\n", + "Gräwe, Deleersnijder, Shah & Heemink (2012). “Why the Euler scheme in particle tracking is not enough: The shallow-sea pycnocline test case.” _Ocean Dynamics_, 62(4), 501–514. https://link.springer.com/article/10.1007/s10236-012-0523-y\n", "\n", "Maruyama, G. (1955). “Continuous Markov processes and stochastic equations.” _Rendiconti del Circolo Matematico di Palermo_, 4(1), 48.\n", "\n", @@ -659,6 +659,11 @@ "\n", "Smagorinsky, J. (1963). “General circulation experiments with primitive equations. 1. The basic experiment.” _Monthly Weather Review_, 91, 99–164. https://doi.org/10.1175/1520-0493(1963)091%3C0099:GCEWTP%3E2.3.CO;2\n" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] } ], "metadata": { @@ -677,7 +682,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.6" + "version": "3.11.5" } }, "nbformat": 4, From be76f2e5e78b9b73cb366aa84b17ee4fab6964f8 Mon Sep 17 00:00:00 2001 From: Erik van Sebille Date: Fri, 3 May 2024 11:31:19 +0200 Subject: [PATCH 3/3] Revert "Updating link to Grawe et al (2012) because doi somehow is broken(?)" This reverts commit f167314743073fe235fa6565ee2fbd40dee42e39. --- docs/examples/tutorial_diffusion.ipynb | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/docs/examples/tutorial_diffusion.ipynb b/docs/examples/tutorial_diffusion.ipynb index 021336479..33f469f4e 100644 --- a/docs/examples/tutorial_diffusion.ipynb +++ b/docs/examples/tutorial_diffusion.ipynb @@ -74,7 +74,7 @@ "\n", "The extra term in the M1 scheme provides extra accuracy at negligible computational cost.\n", "\n", - "The spatial derivatives in the EM and M1 schemes can be approximated by a central difference. Higher order numerical schemes (see [Gräwe et al., 2012](https://link.springer.com/article/10.1007/s10236-012-0523-y)) include higher order derivatives. Since Parcels uses bilinear interpolation, these higher order derivatives cannot be computed, meaning that higher order numerical schemes cannot be used.\n", + "The spatial derivatives in the EM and M1 schemes can be approximated by a central difference. Higher order numerical schemes (see [Gräwe et al., 2012](https://doi.org/10.1007/s10236-012-0523-y)) include higher order derivatives. Since Parcels uses bilinear interpolation, these higher order derivatives cannot be computed, meaning that higher order numerical schemes cannot be used.\n", "\n", "An overview of numerical approximations for SDEs in a particle tracking setting can be found in [Gräwe (2011)](https://doi.org/10.1016/j.ocemod.2010.10.002).\n", "\n", @@ -646,7 +646,7 @@ "\n", "Gräwe, U. (2011). “Implementation of high-order particle-tracking schemes in a water column model.” _Ocean Modelling_, 36(1), 80–89. https://doi.org/10.1016/j.ocemod.2010.10.002\n", "\n", - "Gräwe, Deleersnijder, Shah & Heemink (2012). “Why the Euler scheme in particle tracking is not enough: The shallow-sea pycnocline test case.” _Ocean Dynamics_, 62(4), 501–514. https://link.springer.com/article/10.1007/s10236-012-0523-y\n", + "Gräwe, Deleersnijder, Shah & Heemink (2012). “Why the Euler scheme in particle tracking is not enough: The shallow-sea pycnocline test case.” _Ocean Dynamics_, 62(4), 501–514. https://doi.org/10.1007/s10236-012-0523-y\n", "\n", "Maruyama, G. (1955). “Continuous Markov processes and stochastic equations.” _Rendiconti del Circolo Matematico di Palermo_, 4(1), 48.\n", "\n", @@ -659,11 +659,6 @@ "\n", "Smagorinsky, J. (1963). “General circulation experiments with primitive equations. 1. The basic experiment.” _Monthly Weather Review_, 91, 99–164. https://doi.org/10.1175/1520-0493(1963)091%3C0099:GCEWTP%3E2.3.CO;2\n" ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [] } ], "metadata": { @@ -682,7 +677,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.5" + "version": "3.11.6" } }, "nbformat": 4,