Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for null engine and bp3 option for nvram #1936

Merged
merged 2 commits into from
Jan 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/user_guide/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,8 @@
'engines/hdf5.rst',
'engines/bp3.rst',
'engines/sst.rst',
'engines/inline.rst'
'engines/inline.rst',
'engines/null.rst',
# 'engines/engines.rst',
# 'faq/faq.rst',
]
Expand Down
3 changes: 3 additions & 0 deletions docs/user_guide/source/engines/bp3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ This engine allows the user to fine tune the buffering operations through the fo

9. **SubStreams**: (MPI-only) users can select how many sub-streams (``M`` sub-files) are produced during a run, ranges between 1 and the number of mpi processes from ``MPI_Size`` (``N``), adios2 will internally aggregate data buffers (``N-to-M``) to output the required number of sub-files. If Substream is out of bounds it will pick either 1 (``SubStreams`` < ``1 -> N-to-1``) or ``N`` ((``SubStreams`` > ``N -> N-to-N``) and ADIOS2 will issue a WARNING message. Use for performance tuning.

10. **Node-Local**: For distributed file system. Every writer process must make sure the .bp/ directory is created on the local file system. Required for using local disk/SSD/NVMe in a cluster.

==================== ===================== ===========================================================
**Key** **Value Format** **Default** and Examples
==================== ===================== ===========================================================
Expand All @@ -62,6 +64,7 @@ This engine allows the user to fine tune the buffering operations through the fo
BufferGrowthFactor float > 1 **1.05**, 1.01, 1.5, 2
FlushStepsCount integer > 1 **1**, 5, 1000, 50000
SubStreams integer >= 1 **MPI_Size (N-to-N)**, ``MPI_Size``/2, ... , 2, (N-to-1) 1
Node-Local string On/Off **Off**, On
==================== ===================== ===========================================================


Expand Down
2 changes: 2 additions & 0 deletions docs/user_guide/source/engines/engines.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,5 @@ Parameters are passed at:
.. include:: dataman.rst
.. include:: inline.rst
.. include:: insitu_mpi.rst
.. include:: null.rst
.. include:: virtual_engines.rst
7 changes: 7 additions & 0 deletions docs/user_guide/source/engines/null.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
****
Null
****

The ``Null`` Engine by-passes any heavy I/O operations that other Engines might potentially execute, for example, memory allocations, buffering, transport data movement. Calls to the Null engine would effectively return immediately without doing any effective operations.

The overall goal is to provide a mechanism to isolate an application behavior without the ADIOS 2 footprint. Use this engine to have an idea of the overhead cost of using a certain ADIOS 2 Engine (similar to writing to `/dev/null`) in an application.
7 changes: 3 additions & 4 deletions docs/user_guide/source/engines/virtual_engines.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ The following I/O uses cases are supported by virtual engine names:

1. ``File``: File I/O (Default engine).

This sets up the I/O for files. If the file name passed in Open() ends with ".bp", then the BP4 engine will be used.
This sets up the I/O for files. If the file name passed in Open() ends with ".bp", then the BP4 engine will be used starting in v2.5.0.
If it ends with ".h5", the HDF5 engine will be used. For old .bp files (BP version 3 format), the BP3 engine
will be used for reading.
will be used for reading (v2.4.0 and below).

2. ``FileStream``: Online processing via files.

Expand Down Expand Up @@ -46,8 +46,7 @@ Virtual Engine Setups

These are the actual settings in ADIOS when a virtual engine is selected. The parameters below can be modified before the Open call.

1. ``File``. Engine can be ``BP4`` or ``HDF5`` depending on file extension (".h5" for HDF5, otherwise always BP4),
or it can be ``BP3`` in reading if the input file has an older BP version.
1. ``File``. Refer to the parameter settings for these engines of ``BP4``, ``BP3`` and ``HDF5`` engines earlier in this section.

2. ``FileStream``. The engine is ``BP4``. The parameters are set to:

Expand Down