From 65390e3f15c77d00a171c3033322165b9d5ca508 Mon Sep 17 00:00:00 2001 From: Spiros Tsalikis Date: Thu, 21 Sep 2023 12:29:25 -0400 Subject: [PATCH] Add/Improve the ReadMe.md files in examples directory --- examples/ReadMe.md | 16 +++++ examples/basics/ReadMe.md | 29 ++++++++ examples/basics/queryWorker/ReadMe.md | 11 +++ examples/hello/README.md | 18 ----- examples/hello/ReadMe.md | 69 ++++++++++++++++++ examples/plugins/ReadMe.md | 13 ++++ examples/simulations/GrayScott.jl/ReadMe.md | 71 +++++++++++-------- examples/simulations/ReadMe.md | 30 ++++++++ .../simulations/gray-scott-kokkos/README.md | 2 - .../simulations/gray-scott-kokkos/ReadMe.md | 9 +++ .../gray-scott-struct/CMakeLists.txt | 2 +- .../{README.md => ReadMe.md} | 27 +++---- .../simulations/gray-scott/CMakeLists.txt | 2 +- .../gray-scott/{README.md => ReadMe.md} | 16 ++--- examples/simulations/heatTransfer/ReadMe.md | 70 +++++++++--------- .../simulations/korteweg-de-vries/README.md | 15 ---- .../simulations/korteweg-de-vries/ReadMe.md | 24 +++++++ .../lorenz_ode/{README.md => ReadMe.md} | 13 ++-- examples/useCases/ReadMe.md | 14 ++++ examples/useCases/fidesOneCell/ReadMe.md | 21 +++--- 20 files changed, 334 insertions(+), 138 deletions(-) create mode 100644 examples/ReadMe.md create mode 100644 examples/basics/ReadMe.md create mode 100644 examples/basics/queryWorker/ReadMe.md delete mode 100644 examples/hello/README.md create mode 100644 examples/hello/ReadMe.md create mode 100644 examples/plugins/ReadMe.md create mode 100644 examples/simulations/ReadMe.md delete mode 100644 examples/simulations/gray-scott-kokkos/README.md create mode 100644 examples/simulations/gray-scott-kokkos/ReadMe.md rename examples/simulations/gray-scott-struct/{README.md => ReadMe.md} (93%) rename examples/simulations/gray-scott/{README.md => ReadMe.md} (96%) delete mode 100644 examples/simulations/korteweg-de-vries/README.md create mode 100644 examples/simulations/korteweg-de-vries/ReadMe.md rename examples/simulations/lorenz_ode/{README.md => ReadMe.md} (79%) create mode 100644 examples/useCases/ReadMe.md diff --git a/examples/ReadMe.md b/examples/ReadMe.md new file mode 100644 index 0000000000..b0c91cd9a8 --- /dev/null +++ b/examples/ReadMe.md @@ -0,0 +1,16 @@ +# ADIOS2 Examples + +This directory contains examples of how to use ADIOS2 in different scenarios. +The examples are written mostly in C++, but also C, Fortran, Python, and Julia. + +They can be found in the following subdirectories, and they should be explored in the order that they are listed: + +1. [hello](hello): The _hello_ examples are meant to introduce you to ADIOS2's IO capabilities and engines. +2. [basics](basics): The _basics_ examples are meant to introduce you to basic concepts of ADIOS2, such as + global/joined/local arrays, values, and variables shapes. +3. [useCases](useCases): The _useCases_ examples are meant to demonstrate how to use ADIOS2 in different scenarios, + such as in situ visualization, and fides schema. +4. [plugins](plugins): The _plugins_ examples are meant to introduce you to the plugin capabilities of ADIOS2, such as + how to develop your own engine or operators. +5. [simulations](simulations): The _simulations_ examples are meant to demonstrate how to integrate ADIOS2 within your + simulation code to read, write and/or stream your simulation data. diff --git a/examples/basics/ReadMe.md b/examples/basics/ReadMe.md new file mode 100644 index 0000000000..c33c5545c6 --- /dev/null +++ b/examples/basics/ReadMe.md @@ -0,0 +1,29 @@ +## ADIOS2 basics examples + +The _basics_ examples are meant to introduce you to basic concepts of ADIOS2, such as +global/joined/local arrays, values, and variables shapes. + +They can be found in the following subdirectories, and they should be explored in the order that they are listed: + +1. [globalArray1D](globalArray1D): The _globalArray1D_ example demonstrates how to read and write an + 1-D global array with constant dimensions over time from multiple processors using ADIOS2's BP engine. + * Languages: C, Fortran +2. [globalArrayND](globalArrayND): The _globalArrayND_ example demonstrates how to write an N-D global array with + constant dimensions over time from multiple processors using ADIOS2's BP engine. + * Languages: C++ +3. [localArray](localArray): The _localArray_ example demonstrates how to write and read a local array per processor + with the same name from multiple processors using ADIOS2's BP engine. + * Languages: C++ +4. [joinedArray](joinedArray): The _joinedArray_ example demonstrates how to write local array that is different only in + one dimension so that it can be joined into a global array with the same name from multiple processors at read time + using ADIOS2's ADIOS2's BP engine. + * Languages: C++ +5. [values](values): The _values_ example demonstrates how to write and read a multiple types of variables with a single + value, such as global constant, global value, local constant, and local value using ADIOS2's BP engine. + * Languages: C++, Fortran +6. [variablesShapes](variablesShapes): The _variablesShapes_ example demonstrates how to write supported variables + shapes using stepping and ADIOS2's BP engine. + * Languages: C++, C++ using high-level API +7. [queryWorker](queryWorker): The _queryWorker_ example demonstrates how to read variables using ADIOS2's BP engine + and perform queries on the read data and streams the results. + * Languages: C++ diff --git a/examples/basics/queryWorker/ReadMe.md b/examples/basics/queryWorker/ReadMe.md new file mode 100644 index 0000000000..a241abfefd --- /dev/null +++ b/examples/basics/queryWorker/ReadMe.md @@ -0,0 +1,11 @@ +### ADIOS2 queryWorker example + +The _queryWorker_ example demonstrates how to read variables using ADIOS2's BP engine +and perform queries on the read data and streams the results. + +1. q1.json is a json query file for one var + +2. q2.json is a composite query + +3. The bp*xml are the from heat transfer example with bp4 engine, and tag is added at the end for xml + query spec diff --git a/examples/hello/README.md b/examples/hello/README.md deleted file mode 100644 index e02c03c051..0000000000 --- a/examples/hello/README.md +++ /dev/null @@ -1,18 +0,0 @@ -examples/hello - -Provides tests and illustrates how to use very basic functionality in adios2 - -1. bpWriter - * Write BP format files for one Variable - -1. bpTimeWriter - * Write BP format files for two Variables (one is timestep) using time aggregation - -1. datamanReader (to be deprecated, ADIOS2_USE_DataMan=ON) - * Read real-time WAN streams using dataman - -1. datamanWriter - * Write real-time WAN streams using dataman - -1. hdf5Writer (-DADIOS2_USE_HDF5=ON) - * Write HDF5 files using interoperability through the adios2 interface diff --git a/examples/hello/ReadMe.md b/examples/hello/ReadMe.md new file mode 100644 index 0000000000..90341a3957 --- /dev/null +++ b/examples/hello/ReadMe.md @@ -0,0 +1,69 @@ +## ADIOS2 hello examples + +The _hello_ examples are meant to introduce you to ADIOS2's IO capabilities and engines. + +They can be found in the following subdirectories, and they should be explored in the order that they are listed: + +1. [helloWorld](helloWorld): The _helloWorld_ example demonstrates how to write a simple word and read it back using + ADIOS2's BP engine. + * Languages: C++, C++ using high-level API, C, Python, Python using high-level API +2. [bpWriter](bpWriter): The _bpWriter_ examples demonstrate how to write a variable to using ADIOS2's BP engine. + * Languages: C++, C, Fortran, Python +3. [bpReader](bpReader): The _bpReader_ examples demonstrate how to read a 1D/2D/3D variable using ADIOS2's BP engine. + * Languages: C++, Fortran, Python +4. [bpFWriteCRead](bpFWriteCRead): The _bpFWriteCRead_ example demonstrates how to write a 2D variable with Fortran and + read a subset of it with C++, and vice versa using ADIOS2's BP engine. + * Languages: C++, Fortran +5. [bpTimeWriter](bpTimeWriter): The _bpTimeWriter_ example demonstrates how to write two Variables (one is timestep) + using time aggregation using ADIOS2's BP engine. + * Languages: C++, Python +6. [bpAttributeWriter](bpAttributeWriter): The _bpAttributeWriter_ example demonstrates how to write attributes using + ADIOS2's BP engine. + * Languages: C++ +7. [bpFlushWriter](bpFlushWriter): The _bpFlushWriter_ example demonstrates how to flush a variable using ADIOS2's BP + engine. + * Languages: C++ +8. [bpWriteReadCuda](bpWriteReadCuda): The _bpWriteReadCuda_ example demonstrates how to write and read a variable with + multiple time steps using ADIOS2's BP engine and leveraging CUDA. + * Languages: C++ +9. [bpWriteReadHip](bpWriteReadHip): The _bpWriteReadHip_ example demonstrates how to write and read a variable with + multiple time steps using ADIOS2's BP engine and leveraging HIP. + * Languages: C++ +10. [bpWriteReadKokkos](bpWriteReadKokkos): The _bpWriteReadOmp_ example demonstrates how to write and read a variable + with multiple time steps using ADIOS2's BP engine and leveraging Kokkos. + * Languages: C++ +11. [datamanReader](datamanReader): The _datamanReader_ example demonstrates how to read variables in real-time WAN + streams using ADIOS's DataMan engine. + * Languages: C++, Python +12. [datamanWriter](datamanWriter): The _datamanWriter_ example demonstrates how to write variables in real-time WAN + streams using ADIOS's DataMan engine. + * Languages: C++, Python +13. [dataspacesReader](dataspacesReader): The _dataspacesReader_ example demonstrates how to read a variable using + ADIOS2's DATASPACES engine. + * Languages: C++ +14. [dataspacesWriter](dataspacesWriter): The _dataspacesWriter_ example demonstrates how to write a variable using + ADIOS2's DATASPACES engine. + * Languages: C++ +15. [hdf5Reader](hdf5Reader): The _hdf5Reader_ example demonstrates how to read variables using ADIOS2's HDF5 engine. + * Languages: C++ +16. [hdf5Writer](hdf5Writer): The _hdf5Writer_ example demonstrates how to write variables using ADIOS2's HDF5 engine. + * Languages: C++ +17. [hdf5SubFile](hdf5SubFile): The _hdf5SubFile_ example demonstrates how to write variables using ADIOS2's parallel + HDF5 engine leveraging the subfile feature. + * Languages: C++ +18. [inlineMWE](inlineMWE): The _inlineMWE_ example demonstrates how to write and read a variable using ADIOS2's inline + engine. + * Languages: C++ +19. [inlineFWriteCppRead](inlineFWriteCppRead): The _inlineFWriteCppRead_ example demonstrates how to write a 2D + variable with Fortran and read it back a subset of it with C++ using ADIOS2's inline engine. + * Languages: C++, Fortran +20. [inlineReaderWriter](inlineReaderWriter): The _inlineReaderWriter_ example demonstrates how to write two Variables + (one is timestep) using time aggregation and ADIOS2's inline engine. + * Languages: C++ +21. [sstReader](sstReader): The _sstReader_ example demonstrates how to read a variable using ADIOS2's SST engine. + * Languages: C++ +22. [sstWriter](sstWriter): The _sstWriter_ example demonstrates how to write a variable using ADIOS2's SST engine. + * Languages: C++ +23. [skeleton](skeleton): The _skeleton_ example demonstrates how to write and read a variable using an ADIOS2 skeleton + engine. + * Languages: C++ diff --git a/examples/plugins/ReadMe.md b/examples/plugins/ReadMe.md new file mode 100644 index 0000000000..3fbfdaa0c6 --- /dev/null +++ b/examples/plugins/ReadMe.md @@ -0,0 +1,13 @@ +## ADIOS2 plugins examples + +The _plugins_ examples are meant to introduce you to the plugin capabilities of ADIOS2, such as +how to develop your own engine or operators. + +They can be found in the following subdirectories, and they should be explored in the order that they are listed: + +1. [engine](engine): The _engine_ example demonstrates how to develop the read/write capabilities of your ADIOS2 engine + and use it to read and write variables. + * Languages: C++ +2. [operator](operator): The _operator_ example demonstrates how to read and write variables using the + ADIOS2/plugins/operators/EncryptionOperator and ADIOS2's BP engine. + * Languages: C++ diff --git a/examples/simulations/GrayScott.jl/ReadMe.md b/examples/simulations/GrayScott.jl/ReadMe.md index fe4f08631d..2e12adb5d8 100644 --- a/examples/simulations/GrayScott.jl/ReadMe.md +++ b/examples/simulations/GrayScott.jl/ReadMe.md @@ -1,7 +1,8 @@ -# ADIOS2 GrayScott.jl example +### ADIOS2 GrayScott.jl example -Julia version of [the gray-scott C++ and Python](https://github.com/ornladios/ADIOS2/blob/master/examples/simulations/gray-scott/) -example. +Julia version +of [the gray-scott C++ and Python](https://github.com/ornladios/ADIOS2/blob/master/examples/simulations/gray-scott/) +example. This is a 3D 7-point stencil code to simulate the following [Gray-Scott reaction diffusion model](https://doi.org/10.1126/science.261.5118.189): @@ -11,27 +12,29 @@ u_t = Du * (u_xx + u_yy + u_zz) - u * v^2 + F * (1 - u) + noise * randn(-1,1) v_t = Dv * (v_xx + v_yy + v_zz) + u * v^2 - (F + k) * v ``` -This version contains: +This version contains: - CPU threaded solver using Julia's [multithreading]](https://docs.julialang.org/en/v1/manual/multi-threading/) - GPU solvers using [CUDA.jl](https://github.com/JuliaGPU/CUDA.jl), [AMDGPU.jl](https://github.com/JuliaGPU/AMDGPU.jl) -- Parallel I/O using the [ADIOS2.jl](https://github.com/eschnett/ADIOS2.jl) Julia bindings to [ADIOS2](https://github.com/ornladios/ADIOS2) +- Parallel I/O using the [ADIOS2.jl](https://github.com/eschnett/ADIOS2.jl) Julia bindings + to [ADIOS2](https://github.com/ornladios/ADIOS2) - Message passing interface (MPI) using [MPI.jl](https://github.com/JuliaParallel/MPI.jl) Julia bindings to MPI - Easily switch between float- (Float32) and double- (Float64) precision in the configuration file -## How to run +#### How to run -Currently only the simulation part is ported from C++, the data analysis is work-in-progress. +Currently only the simulation part is ported from C++, the data analysis is work-in-progress. Pre-requisites: -- A recent Julia version: v1.8.5 or v1.9.0-beta3 as of January 2023 from [julialang.org/downloads](https://julialang.org/downloads/) +- A recent Julia version: v1.8.5 or v1.9.0-beta3 as of January 2023 + from [julialang.org/downloads](https://julialang.org/downloads/) -### Run locally +##### Run locally 1. **Set up dependencies** -From the `GrayScott.jl` directory instantiate and use MPI artifact jll (preferred method). +From the `GrayScott.jl` directory instantiate and use MPI artifact jll (preferred method). To use a system provided MPI, see [here](https://juliaparallel.org/MPI.jl/latest/configuration/#using_system_mpi) ```julia @@ -50,7 +53,8 @@ julia> MPIPreferences.use_jll_binary() julia> exit() ``` -Julia manages its own packages using [Pkg.jl](https://pkgdocs.julialang.org/v1/), the above would create platform-specific `LocalPreferences.toml` and `Manifest.toml` files. +Julia manages its own packages using [Pkg.jl](https://pkgdocs.julialang.org/v1/), the above would create +platform-specific `LocalPreferences.toml` and `Manifest.toml` files. 2. **Set up the examples/settings-files.json configuration file** @@ -81,8 +85,8 @@ Julia manages its own packages using [Pkg.jl](https://pkgdocs.julialang.org/v1/) ``` -The file is nearly identical to the C++ original example. -Not all options are currently supported, but two Julia-only options are added: +The file is nearly identical to the C++ original example. +Not all options are currently supported, but two Julia-only options are added: - "precision": either Float32 or Float64 in the array simulation (including GPUs) - "backend": "CPU", "CUDA" or "AMDGPU" @@ -103,37 +107,48 @@ Not all options are currently supported, but two Julia-only options are added: This would generate an adios2 file from the output entry in the configuration file (e.g. `gs-julia-1MPI-64L-F32.bp`) that can be visualized with ParaView with either the VTX or the FIDES readers. -**Important**: the AMDGPU.jl implementation of `randn` is currently work in progress. +**Important**: the AMDGPU.jl implementation of `randn` is currently work in progress. See related issue [here](https://github.com/JuliaGPU/AMDGPU.jl/issues/378) - 4. **Running on OLCF Summit and Crusher systems** -The code was tested on the Oak Ridge National Laboratory Leadership Computing Facilities (OLCF): [Summit](https://docs.olcf.ornl.gov/systems/summit_user_guide.html) and [Crusher](https://docs.olcf.ornl.gov/systems/crusher_quick_start_guide.html). Both are used testing a recent version of Julia [v1.9.0-beta3](https://julialang.org/downloads/#upcoming_release) and a `JULIA_DEPOT_PATH` is required to install packages and artifacts. **DO NOT USE your home directory**. We are providing configuration scripts in `scripts/config_XXX.sh` showing the plumming required for these systems. They need to be executed only once per session from the login nodes. - -To reuse these file the first 3 entries must be modified and run on login-nodes and the PATH poiting at a downloaded Julia binary for the corresponding PowerPC (Summit) and x86-64 (Crusher) architectures. Only "CPU" and "CUDA" backends are supported on Summit, while "CPU" and "AMDGPU" backends are supported on Crusher. + The code was tested on the Oak Ridge National Laboratory Leadership Computing Facilities ( + OLCF): [Summit](https://docs.olcf.ornl.gov/systems/summit_user_guide.html) + and [Crusher](https://docs.olcf.ornl.gov/systems/crusher_quick_start_guide.html). Both are used testing a recent + version of Julia [v1.9.0-beta3](https://julialang.org/downloads/#upcoming_release) and a `JULIA_DEPOT_PATH` is + required to install packages and artifacts. **DO NOT USE your home directory**. We are providing configuration + scripts in `scripts/config_XXX.sh` showing the plumming required for these systems. They need to be executed only + once per session from the login nodes. + +To reuse these file the first 3 entries must be modified and run on login-nodes and the PATH poiting at a downloaded +Julia binary for the corresponding PowerPC (Summit) and x86-64 (Crusher) architectures. Only "CPU" and "CUDA" backends +are supported on Summit, while "CPU" and "AMDGPU" backends are supported on Crusher. ``` - # Replace these 3 entries + ### Replace these 3 entries PROJ_DIR=/gpfs/alpine/proj-shared/csc383 export JULIA_DEPOT_PATH=$PROJ_DIR/etc/summit/julia_depot GS_DIR=$PROJ_DIR/wgodoy/ADIOS2/examples/simulations/GrayScott.jl ... - # and the path + ### and the path export PATH=$PROJ_DIR/opt/summit/julia-1.9.0-beta3/bin:$PATH ``` -## To-do list +#### To-do list - 1. Add support including random number on device kernel code on `AMDGPU.jl` - 2. Set the domain size `L` in the configuration file as a multiple of 6 for Summit, and a multiple of 4 on Crusher - 3. Add data analysis: PDF for u and v and Julia 2D plotting capabilities: Plots.jl, Makie.jl - 4. Add interactive computing with Pluto.jl notebooks +1. Add support including random number on device kernel code on `AMDGPU.jl` +2. Set the domain size `L` in the configuration file as a multiple of 6 for Summit, and a multiple of 4 on Crusher +3. Add data analysis: PDF for u and v and Julia 2D plotting capabilities: Plots.jl, Makie.jl +4. Add interactive computing with Pluto.jl notebooks +#### Acknowledgements -## Acknowledgements -This research was supported by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department of Energy’s Office of Science and National Nuclear Security Administration, responsible for delivering a capable exascale ecosystem, including software, applications, and hardware technology, to support the nation’s exascale computing imperative. +This research was supported by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department of +Energy’s Office of Science and National Nuclear Security Administration, responsible for delivering a capable exascale +ecosystem, including software, applications, and hardware technology, to support the nation’s exascale computing +imperative. -This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. +This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which +is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. Thanks to the Exascale Computing Project PROTEAS-TUNE and ADIOS subprojects, and the ASCR Bluestone. Thanks to all the Julia community members, packages developers and maintainers for their great work. \ No newline at end of file diff --git a/examples/simulations/ReadMe.md b/examples/simulations/ReadMe.md new file mode 100644 index 0000000000..d3b72b36a6 --- /dev/null +++ b/examples/simulations/ReadMe.md @@ -0,0 +1,30 @@ +## ADIOS2 simulations examples + +The _simulations_ examples are meant to demonstrate how to integrate ADIOS2 within your +simulation code to read, write and/or stream your simulation data. + +They can be found in the following subdirectories, and they should be explored in the order that they are listed: + +1. [korteweg-de-vries](korteweg-de-vries): The _korteweg-de-vries_ example demonstrates how to solve the initial value + problem for the Korteweg de-Vries equation via the Zabusky and Krustal scheme and integrate ADIOS2 for its IO + capabilities. + * Languages: C++, Python +2. [lorenz_ode](lorenz_ode): The _lorenz_ode_ example demonstrates how to solve the Lorenz system of ordinary + differential equations and integrate ADIOS2 for its IO capabilities. + * Languages: C++ +3. [gray-scott](gray-scott): The _gray-scott_ example demonstrates how to perform simulation with the Gray-Scott + reaction diffusion model and integrate ADIOS2 for its IO capabilities. + * Languages: C++ +4. [gray-scott-struct](gray-scott-struct): The _gray-scott-struct_ example demonstrates how to perform a simulation with + the Gray-Scott reaction diffusion model using a struct that defines user defined data types and integrate ADIOS2 for + its IO capabilities. + * Languages: C++ +5. [gray-scott-kokkos](gray-scott-kokkos): The _gray-scott-kokkos_ example demonstrates how to perform a simulation with + the Gray-Scott reaction diffusion model using Kokkos and integrate ADIOS2 for its IO capabilities. + * Languages: C++ +6. [GrayScott.jl](GrayScott.jl): The _GrayScott.jl_ example demonstrates how to perform a simulation with the Gray-Scott + reaction diffusion model and integrate ADIOS2 for its IO capabilities. + * Languages: Julia +7. [heatTransfer](heatTransfer): The _heatTransfer_ example demonstrates how to solve a 2D Poisson equation for + temperature in homogeneous media using finite differences and integrate ADIOS2 for its IO capabilities. + * Languages: C++ diff --git a/examples/simulations/gray-scott-kokkos/README.md b/examples/simulations/gray-scott-kokkos/README.md deleted file mode 100644 index 24f6213b5e..0000000000 --- a/examples/simulations/gray-scott-kokkos/README.md +++ /dev/null @@ -1,2 +0,0 @@ -Kokkos version of Gray-Scott. -Please use the installed Gray-Scott example directory but run adios2-gray-scott-kokkos binary. diff --git a/examples/simulations/gray-scott-kokkos/ReadMe.md b/examples/simulations/gray-scott-kokkos/ReadMe.md new file mode 100644 index 0000000000..86790505d0 --- /dev/null +++ b/examples/simulations/gray-scott-kokkos/ReadMe.md @@ -0,0 +1,9 @@ +### ADIOS2 gray-scott-kokkos example + +This is a 3D 7-point stencil code to simulate the following [Gray-Scott +reaction diffusion model](https://doi.org/10.1126/science.261.5118.189): + +#### Differences from the gray-scott example + +It uses Kokkos to run the simulation on GPU. +Please use the installed Gray-Scott example directory but run adios2_simulations_gray-scott-kokkos binary. diff --git a/examples/simulations/gray-scott-struct/CMakeLists.txt b/examples/simulations/gray-scott-struct/CMakeLists.txt index 4d490eca3f..a892b757dd 100644 --- a/examples/simulations/gray-scott-struct/CMakeLists.txt +++ b/examples/simulations/gray-scott-struct/CMakeLists.txt @@ -42,7 +42,7 @@ if(ADIOS2_HAVE_MPI) "simulation/settings-staging.json" "simulation/settings-inline.json" "plot/decomp.py" "plot/gsplot.py" "plot/pdfplot.py" - "README.md" + "ReadMe.md" DESTINATION ${CMAKE_INSTALL_PREFIX}/share/adios2/examples/simulations/gray-scott-struct) install(DIRECTORY "catalyst" DESTINATION ${CMAKE_INSTALL_PREFIX}/share/adios2/examples/simulations/gray-scott-struct) diff --git a/examples/simulations/gray-scott-struct/README.md b/examples/simulations/gray-scott-struct/ReadMe.md similarity index 93% rename from examples/simulations/gray-scott-struct/README.md rename to examples/simulations/gray-scott-struct/ReadMe.md index 774523c1d2..b82eaa279b 100644 --- a/examples/simulations/gray-scott-struct/README.md +++ b/examples/simulations/gray-scott-struct/ReadMe.md @@ -1,19 +1,20 @@ -# ADIOS2 gray-scott-struct example -# -# This is a version where the computation is on a struct memory type holding U and V -# instead of having two separate arrays for U and V. It was created for the purpose of -# testing user defined data types in ADIOS2 -# +### ADIOS2 gray-scott-struct example This is a 3D 7-point stencil code to simulate the following [Gray-Scott reaction diffusion model](https://doi.org/10.1126/science.261.5118.189): +#### Differences from the gray-scott example + +This is a version where the computation is on a struct memory type holding U and V +instead of having two separate arrays for U and V. It was created for the purpose of +testing user defined data types in ADIOS2 + ``` u_t = Du * (u_xx + u_yy + u_zz) - u * v^2 + F * (1 - u) v_t = Dv * (v_xx + v_yy + v_zz) + u * v^2 - (F + k) * v ``` -## How to run +#### How to run Make sure MPI and ADIOS2 are installed and that the `PYTHONPATH` includes the ADIOS2 package. Make sure the adios2/bin installation directory is in the `PATH` (conda and spack installations should take care of this @@ -57,7 +58,7 @@ $ python3 gsplot.py -i gs.bp ``` -## Analysis example how to run +#### Analysis example how to run ``` $ mpirun -n 4 adios2_simulations_gray-scott-struct settings-files.json @@ -77,7 +78,7 @@ Each process plots the middle slice of their subarray U/pdf[x:y,:] ``` -## How to change the parameters +#### How to change the parameters Edit settings.json to change the parameters for the simulation. @@ -97,7 +98,7 @@ Edit settings.json to change the parameters for the simulation. Decomposition is automatically determined by MPI_Dims_create. -## Examples +#### Examples | D_u | D_v | F | k | Output | |-----|-----|------|--------|--------------------------------| @@ -107,7 +108,7 @@ Decomposition is automatically determined by MPI_Dims_create. | 0.2 | 0.1 | 0.01 | 0.05 | ![](img/example4.jpg?raw=true) | | 0.2 | 0.1 | 0.02 | 0.06 | ![](img/example5.jpg?raw=true) | -## In situ pipeline example +#### In situ pipeline example In adios2.xml, change all IO groups' engine to SST. @@ -131,7 +132,7 @@ $ mpirun -n 4 adios2_simulations_gray-scott-struct settings-staging.json : \ -n 1 python3 gsplot.py -i gs.bp ``` -## In situ batch visualization with ParaView Catalyst +#### In situ batch visualization with ParaView Catalyst This requires ADIOS 2.8.0 or later, due to the use of `ParaViewFidesEngine` plugin for ADIOS. Internally, this plugin uses the ADIOS inline engine to pass data pointers to ParaView's @@ -154,7 +155,7 @@ The `gs-pipeline.py` contains the pipeline Catalyst will execute on each step. These files are passed as parameters to the engine plugin (see parameters `DataModel` and `Script` in the `adios2-inline-plugin.xml` file). -### Build and Run +##### Build and Run This example is built as normal (making sure you are using ADIOS v2.8.0 or later) and does not have build dependencies on ParaView. diff --git a/examples/simulations/gray-scott/CMakeLists.txt b/examples/simulations/gray-scott/CMakeLists.txt index a5fc1745ac..7fed3add60 100644 --- a/examples/simulations/gray-scott/CMakeLists.txt +++ b/examples/simulations/gray-scott/CMakeLists.txt @@ -42,7 +42,7 @@ if(ADIOS2_HAVE_MPI) "simulation/settings-staging.json" "simulation/settings-inline.json" "plot/decomp.py" "plot/gsplot.py" "plot/pdfplot.py" - "README.md" + "ReadMe.md" DESTINATION ${CMAKE_INSTALL_PREFIX}/share/adios2/examples/simulations/gray-scott) install(DIRECTORY "catalyst" DESTINATION ${CMAKE_INSTALL_PREFIX}/share/adios2/examples/simulations/gray-scott) diff --git a/examples/simulations/gray-scott/README.md b/examples/simulations/gray-scott/ReadMe.md similarity index 96% rename from examples/simulations/gray-scott/README.md rename to examples/simulations/gray-scott/ReadMe.md index 94b672d769..db615b1f12 100644 --- a/examples/simulations/gray-scott/README.md +++ b/examples/simulations/gray-scott/ReadMe.md @@ -1,4 +1,4 @@ -# ADIOS2 gray-scott example +### ADIOS2 gray-scott example This is a 3D 7-point stencil code to simulate the following [Gray-Scott reaction diffusion model](https://doi.org/10.1126/science.261.5118.189): @@ -8,7 +8,7 @@ u_t = Du * (u_xx + u_yy + u_zz) - u * v^2 + F * (1 - u) + noise * randn(-1,1) v_t = Dv * (v_xx + v_yy + v_zz) + u * v^2 - (F + k) * v ``` -## How to run +#### How to run Make sure MPI and ADIOS2 are installed and that the `PYTHONPATH` includes the ADIOS2 package. Make sure the adios2/bin installation directory is in the `PATH` (conda and spack installations should take @@ -52,7 +52,7 @@ $ python3 gsplot.py -i gs.bp ``` -## Analysis example how to run +#### Analysis example how to run ``` $ mpirun -n 4 adios2_simulations_gray-scott settings-files.json @@ -72,7 +72,7 @@ Each process plots the middle slice of their subarray U/pdf[x:y,:] ``` -## How to change the parameters +#### How to change the parameters Edit settings.json to change the parameters for the simulation. @@ -92,7 +92,7 @@ Edit settings.json to change the parameters for the simulation. Decomposition is automatically determined by MPI_Dims_create. -## Examples +#### Examples | D_u | D_v | F | k | Output | |-----|-----|------|--------|--------------------------------| @@ -126,7 +126,7 @@ $ mpirun -n 4 adios2_simulations_gray-scott settings-staging.json : \ -n 1 python3 gsplot.py -i gs.bp ``` -## In situ batch and interactive visualization with ParaView Catalyst +#### In situ batch and interactive visualization with ParaView Catalyst This requires ADIOS 2.9.0 or later, due to the use of `ParaViewADIOSInSituEngine` plugin for ADIOS. Internally, this plugin uses the ADIOS inline engine to pass data pointers to ParaView's @@ -149,7 +149,7 @@ The `gs-pipeline.py` contains the pipeline Catalyst will execute on each step. These files are passed as parameters to the engine plugin (see parameters `DataModel` and `Script` in the `adios2-inline-plugin.xml` file). -### Build and Run +##### Build and Run This example is built as normal (making sure you are using ADIOS v2.9.0 or later) and does not have build dependencies on ParaView. @@ -197,7 +197,7 @@ To run: $ mpirun -n 4 build/adios2_simulations_gray-scott simulation/settings-inline.json ``` -### Interactive visualization +##### Interactive visualization Open the ParaView GUI. `/path/to/paraview/build/bin/paraview` On the `Catalyst` menu, click `Connect`. You can leave the default port of 22222. diff --git a/examples/simulations/heatTransfer/ReadMe.md b/examples/simulations/heatTransfer/ReadMe.md index a378c72f96..8693d00265 100644 --- a/examples/simulations/heatTransfer/ReadMe.md +++ b/examples/simulations/heatTransfer/ReadMe.md @@ -1,33 +1,30 @@ -### heatTransfer example +### ADIOS2 heatTransfer example This example solves a 2D Poisson equation for temperature in homogeneous media -using finite differences. This examples shows a straight-forward way to hook +using finite differences. This examples shows a straight-forward way to hook an application to the ADIOS2 library for its IO. - 1. write: illustrates the Write API as well as has implementations of other IO libraries - - * adios 2 - * hdf5 sequential, separate file per process per step - * phdf5 parallel, steps appended to the same one file + * adios 2 + * hdf5 sequential, separate file per process per step + * phdf5 parallel, steps appended to the same one file 2. read: illustrates the Read API that allows running the reader either as + * post-mortem to read all output steps + * in situ to read step by step as the writer outputs them (need to run with a suitable engine) - * post-mortem to read all output steps - * in situ to read step by step as the writer outputs them - (need to run with a suitable engine) - -3. readFileOnly: illustrates reading all output steps at once (a single read - statement) into a single contiguous memory block. This approach only works - for post-mortem processing. - - +3. readFileOnly: illustrates reading all output steps at once (a single read + statement) into a single contiguous memory block. This approach only works + for post-mortem processing. -Example +4. inline: illustrates the inline engine that allows the reader to read the + data as it is being written. +#### Example -1. Produce an output +##### 1. Produce an output +``` Writer usage: heatTransfer config output N M nx ny steps iterations config: XML config file to use output: name of output data file/stream @@ -37,37 +34,44 @@ Writer usage: heatTransfer config output N M nx ny steps iterations ny: local array size in Y dimension per processor steps: the total number of steps to output iterations: one step consist of this many iterations +``` -The ADIOS2 executable needs an XML config file to select the Engine used for the output. The engines are: File, BP4 and HDF5, the corresponding XML config files are in the examples/simulations/heatTransfer/ directory. The "File" engine will be BP4 or HDF5 depending on the extension of the file name. +The ADIOS2 executable needs an XML config file to select the Engine used for the output. The engines are: File, BP4 and +HDF5, the corresponding XML config files are in the examples/simulations/heatTransfer/ directory. The "File" engine will +be BP4 or HDF5 depending on the extension of the file name. -The adios1, ph5 and hdf5 versions of the example do not use XML config files, so just type "none" for the config argument. +The adios1, ph5 and hdf5 versions of the example do not use XML config files, so just type "none" for the config +argument. -$ mpirun -np 12 ./bin/adios2_simulations_heatTransferWrite ../examples/simulations/heatTransfer/heat_file.xml heat.bp 4 3 5 10 10 10 -$ mpirun -np 12 ./bin/adios2_simulations_heatTransferWrite ../examples/simulations/heatTransfer/heat_file.xml heat.h5 4 3 5 10 10 10 +``` +$ mpirun -np 12 ./bin/adios2_simulations_heatTransferWrite ../examples/simulations/heatTransfer/heat_file.xml heat.bp 4 +3 5 10 10 10 +$ mpirun -np 12 ./bin/adios2_simulations_heatTransferWrite ../examples/simulations/heatTransfer/heat_file.xml heat.h5 4 +3 5 10 10 10 +``` +##### 2. Read the output step-by-step and print data into text files (data. per reader process) -2. Read the output step-by-step and print data into text files (data. per reader process) - -Reader Usage: heatRead config input N M +``` +Reader Usage: heatRead config input N M config: XML config file to use input: name of input data file/stream N: number of processes in X dimension M: number of processes in Y dimension +``` +``` +$ mpirun -np 2 ./bin/heatTransfer_read ../examples/simulations/heatTransfer/heat_file.xml heat 2 1 +``` -$ mpirun -np 2 ./bin/heatTransfer_read ../examples/simulations/heatTransfer/heat_file.xml heat 2 1 - - -Notes: -1. Engines for file-based output and post-mortem reading: i +##### Notes: +1. Engines for file-based output and post-mortem reading: i * BPFileWriter/BPFileReader * HDF5Writer/HDF5Reader 2. Engines for in situ execution - - * DataManWriter/DataManReader - (Must run writer and reader with the same number of processes and same decomposition) + * DataManWriter/DataManReader (Must run writer and reader with the same number of processes and same decomposition) diff --git a/examples/simulations/korteweg-de-vries/README.md b/examples/simulations/korteweg-de-vries/README.md deleted file mode 100644 index 42f67fac7e..0000000000 --- a/examples/simulations/korteweg-de-vries/README.md +++ /dev/null @@ -1,15 +0,0 @@ -To run: - -``` -ADIOS2/build$ ./bin/kdv -``` - -To view data generated: - -``` -$ conda create -n kdv -$ conda activate kdv -$ conda install -c williamfgc adios2 -c conda-forge -$ conda install pip -$ pip install diagram -``` diff --git a/examples/simulations/korteweg-de-vries/ReadMe.md b/examples/simulations/korteweg-de-vries/ReadMe.md new file mode 100644 index 0000000000..d172ff05f7 --- /dev/null +++ b/examples/simulations/korteweg-de-vries/ReadMe.md @@ -0,0 +1,24 @@ +### ADIOS2 korteweg-de-vries example + +Solves the initial value problem for the Korteweg de-Vries equation via the Zabusky and Krustal scheme. + +#### Running the example + +``` +./adios2_simulations_kortewegDeVries N t_max δ + N is number of spacial gridpoints (∆t chosen from ∆x via the stability condition) + t_max is max simulation time + δ is an interaction parameter; + +e.g., ./adios2_simulations_kortewegDeVries 512 10 0.022 +``` + +#### View data generated: + +``` +$ conda create -n adios2_simulations_kortewegDeVries +$ conda activate adios2_simulations_kortewegDeVries +$ conda install -c williamfgc adios2 -c conda-forge +$ conda install pip +$ pip install diagram +``` diff --git a/examples/simulations/lorenz_ode/README.md b/examples/simulations/lorenz_ode/ReadMe.md similarity index 79% rename from examples/simulations/lorenz_ode/README.md rename to examples/simulations/lorenz_ode/ReadMe.md index 7f18b766dc..f01f7aca84 100644 --- a/examples/simulations/lorenz_ode/README.md +++ b/examples/simulations/lorenz_ode/ReadMe.md @@ -1,8 +1,10 @@ -## Lorenz System Example +### ADIOS2 Lorenz System Example -The [Lorenz system](https://en.wikipedia.org/wiki/Lorenz_system) is a very simple ODE which exhibits large sensitivity to initial conditions. +The [Lorenz system](https://en.wikipedia.org/wiki/Lorenz_system) is a very simple ODE which exhibits large sensitivity +to initial conditions. -This example exhibits a number of capabilities of ADIOS: For example, how do we write data that is completely independent of other data rapidly? +This example exhibits a number of capabilities of ADIOS: For example, how do we write data that is completely +independent of other data rapidly? How do we write complex data structures that are not naturally layed out as matrices? The ODE to be solved is @@ -28,9 +30,8 @@ but we will be able to store less total data if we store the phase-space state ![Alt text](./ode_phase_space.svg) and use cubic Hermite interpolation for rendering. -In addition, it doesn't make much sense to *solve* our equation more accurately than we render it, so we'll simply use the second derivative to pick stepsizes. - - +In addition, it doesn't make much sense to *solve* our equation more accurately than we render it, so we'll simply use +the second derivative to pick stepsizes. A natural C++ data structure for this is a `std::vector>`. This will give us a nontrivial data structure to examine in ADIOS2. diff --git a/examples/useCases/ReadMe.md b/examples/useCases/ReadMe.md new file mode 100644 index 0000000000..d49a91ffe1 --- /dev/null +++ b/examples/useCases/ReadMe.md @@ -0,0 +1,14 @@ +## ADIOS2 useCases examples + +The _useCases_ examples are meant to demonstrate how to use ADIOS2 in different scenarios, +such as in situ visualization, and fides schema. + +They can be found in the following subdirectories, and they should be explored in the order that they are listed: + +1. [insituGlobalArrays](insituGlobalArrays): The _insituGlobalArrays_ example demonstrates how to write and read a + global array with constant dimensions over time from multiple processors using ADIOS2's SST, insitumpi or DataMan + engines. + * Languages: C++ +2. [fidesOneCell](fidesOneCell): The _fidesOneCell_ example demonstrates how to write a hexagon using ADIOS2's BP engine, and + provides a fides schema for visualizing it in ParaView. + * Languages: C++ diff --git a/examples/useCases/fidesOneCell/ReadMe.md b/examples/useCases/fidesOneCell/ReadMe.md index ef39aa28e0..a6df79a0a8 100644 --- a/examples/useCases/fidesOneCell/ReadMe.md +++ b/examples/useCases/fidesOneCell/ReadMe.md @@ -1,16 +1,14 @@ -Single cell example for visualization with Fides schema in ParaView -------------------------------------------------------------------- +### Single cell example for visualization with Fides schema in ParaView This example writes 8 points in 3D space, the corners of a box, and then defines a single hexagon cell of those eight -points. -The coordinates of the points are stored in three 1D arrays, separating the X-Y-Z coordinates. +points. The coordinates of the points are stored in three 1D arrays, separating the X-Y-Z coordinates. The cell is the enumeration of the points connected in the order of a hexagon prescribed by VTK, see -Figure 19-20 in the VTK User Guide, type VTK_HEXAHEDRON in page 480 (page 494 in the PDF). -https://vtk.org/wp-content/uploads/2021/08/VTKUsersGuide.pdf +Figure 19-20 in the [VTK User Guide](https://vtk.org/wp-content/uploads/2021/08/VTKUsersGuide.pdf), type VTK_HEXAHEDRON +in page 480 (page 494 in the PDF). -The visualization schema is described in the fidesOneCell.json file, according -to https://fides.readthedocs.io/en/latest/schema/schema.html +The fides visualization schema is described in the fidesOneCell.json file can be +found [here](https://fides.readthedocs.io/en/latest/schema/schema.html). We compose the coordinate system as a composite of the three 1D arrays for X, Y and Z coordinates. This is just an enumeration of the N points in space (N = size of X = size of Y = size of Z) , in contrast to other examples where X, Y, @@ -27,11 +25,8 @@ DataOnPoints array from the file). First step of onecell.bp in ParaView: -paraview-onecell-step0.png +![paraview-onecell-step0.png](paraview-onecell-step0.png) Last step of onecell.bp in ParaView: -paraview-onecell-step9.png - +![paraview-onecell-step9.png](paraview-onecell-step9.png)