readme.md

MPI

Modern C++20 message passing interface wrapper.

Examples

• Initialization:

  mpi::environment environment;
  const auto& communicator = mpi::world_communicator;

• Transmitting basic types:

  std::int32_t data = 0;
  if (communicator.rank() == 0)
  {
    data = 42;
    communicator.send(data, 1);
  }
  if (communicator.rank() == 1)
  {
    communicator.receive(data, 0);
  }

• Transmitting a container of basic types:

  std::vector<std::int32_t> data_container(3);
  if (communicator.rank() == 0)
  {
    data_container = {1, 2, 3};
    communicator.send(data_container, 1);
  }
  if (communicator.rank() == 1)
  {
    communicator.receive(data_container, 0);
  }

• Transmitting user-defined aggregates:

  struct user_type
  {
    std::int32_t         id      ;
    std::array<float, 3> position;
  };

  user_type user_object;
  if (communicator.rank() == 0)
  {
    user_object = {42, {0.0f, 1.0f, 2.0f}};
    communicator.send(user_object, 1);
  }
  if (communicator.rank() == 1)
  {
    communicator.receive(user_object, 0);
  }

• Transmitting a container of user-defined aggregates:

  std::vector<user_type> user_object_container(2);
  if (communicator.rank() == 0)
  {
    user_object_container[0] = {42, {0.0f, 1.0f, 2.0f}};
    user_object_container[1] = {84, {3.0f, 4.0f, 5.0f}};
    communicator.send(user_object_container, 1);
  }
  if (communicator.rank() == 1)
  {
    communicator.receive(user_object_container, 0);
  }

• See the tests for more.

Usage Notes

• Define MPI_USE_EXCEPTIONS to check the return values of all viable functions against MPI_SUCCESS and throw an exception otherwise.
• Define MPI_USE_RELAXED_TRAITS to prevent the library from checking the types of aggregate elements and triggering static asserts for non-aggregates (useful for e.g. testing).
• Define MPI_USE_UNSUPPORTED to enable features that are within the standard but currently not supported by major implementations (e.g. MPI_T_BIND_MPI_SESSION).
• Compliant types (satisfying mpi::is_compliant) are types whose corresponding mpi::data_type can be automatically generated:
  • Arithmetic types (satisfying std::is_arithmetic), enumerations (satisfying std::is_enum), specializations of std::complex are compliant types.
  • C-style arrays, std::array, std::pair, std::tuple, and aggregate types (satisfying std::is_aggregate) consisting of other compliant types are also compliant types.
  • If your type is none of the above, you can specialize template <> struct mpi::type_traits<TYPE> { static data_type get_data_type() { return DATA_TYPE; } }; manually.
• The MPI functions accepting buffers may be used with:
  • Compliant types.
  • Contiguous sequential containers (i.e. std::string, std::span, std::valarray, std::vector<!bool>) of compliant types.
• Extension functions (starting with MPIX) are not included as they are often implementation-specific. You can nevertheless use them with the wrapper via the native handle getters.

Design Notes

• Constructors:
  • Copy constructors are deleted unless MPI provides duplication functions (ending with _dup) for the object.
  • Move constructors are available whenever possible.
  • The object wrappers have two types of constructors:
    • Managed constructors: Construct a new MPI object, and be responsible for its destruction.
    • Unmanaged constructors: Accept an existing MPI object, and not be responsible for its destruction.
• Structures:
  • Structs are for POD data types. They may only contain member variables, constructors, destructors and assignment operators.
• Statics:
  • Namespace functions are preferred over static member functions whenever possible.
• Reflection:
  • The data type for aggregates are created automatically through reflection.
  • Survey of reflection libraries:

    Library	Built	Extra step	Manual	Dependencies
    https://github.com/rttrorg/rttr	True	False	True	False
    https://github.com/boostorg/pfr	False	False	False	False
    https://github.com/apolukhin/pfr_non_boost	False	False	False	False
    https://github.com/felixguendling/cista	False	False	False	False
    https://github.com/veselink1/refl-cpp	False	False	True	False
    https://github.com/AustinBrunkhorst/CPP-Reflection	True	True	True	True
    https://github.com/billyquith/ponder	True	False	True	False
    https://github.com/skypjack/meta	False	False	True	False
    https://github.com/Ubpa/USRefl	False	False	True	False
    https://github.com/Ubpa/UDRefl	True	False	True	False
    https://github.com/garbageslam/visit_struct	False	False	True	False
    https://github.com/chakaz/reflang	True	True	False	True
    https://github.com/Manu343726/tinyrefl	False	True	False	False
    https://github.com/Manu343726/siplasplas	True	True	False	False
    https://github.com/preshing/FlexibleReflection	False	False	True	False
    https://github.com/Celtoys/clReflect	True	True	True	False
    https://github.com/Leandros/metareflect	False	True	True	True
    https://github.com/Cylix/Reflex	False	False	True	True
    https://github.com/boostorg/describe	False	False	True	True

  • Decided on PFR after watching https://www.youtube.com/watch?v=abdeAew3gmQ, specifically https://github.com/apolukhin/pfr_non_boost without the boost namespace.
  • Limitations:
    • The type must be an aggregate (satisfy std::is_aggregate<type>):
      • No user-declared or inherited constructors.
      • No private or protected direct non-static data members.
      • No virtual functions.
      • No virtual, private or protected base classes.
    • Additionally:
      • No const fields.
      • No references.
      • Static data members are ignored.
    • See https://www.boost.org/doc/libs/1_76_0/doc/html/boost_pfr/limitations_and_configuration.html for further detail.
• Constants:
  • Several MPI constants are unused:
    • MPI_STATUS_IGNORE : Stati are return values and this would require void overloads for every function.
    • MPI_STATUSES_IGNORE: Stati are return values and this would require void overloads for every function.
    • MPI_ARGVS_NULL : This would require a scan over std::vector<spawn_info>::arguments() to see if they all are empty.
    • MPI_WEIGHTS_EMPTY : Prefer an empty container instead.
    • MPI_BOTTOM : Prefer nullptr instead.
    • MPI_ROOT : Unused in default arguments as intercommunicators are rarer than intracommunicators. Can nevertheless be passed to functions.
    • MPI_PROC_NULL : Unused in default arguments as intercommunicators are rarer than intracommunicators. Can nevertheless be passed to functions.
• Versioning:
  • The major and minor version numbers correspond to the supported MPI version, whereas the patch number is specific to the wrapper.

Coverage (list from https://www.open-mpi.org/doc/v4.1/)

• Constants
• MPI_Abort
• MPI_Accumulate
• MPI_Add_error_class
• MPI_Add_error_code
• MPI_Add_error_string
• MPI_Address
• MPI_Aint_add
• MPI_Aint_diff
• MPI_Allgather
• MPI_Allgatherv
• MPI_Alloc_mem
• MPI_Allreduce
• MPI_Alltoall
• MPI_Alltoallv
• MPI_Alltoallw
• MPI_Attr_delete
• MPI_Attr_get
• MPI_Attr_put
• MPI_Barrier
• MPI_Bcast
• MPI_Bsend
• MPI_Bsend_init
• MPI_Buffer_attach
• MPI_Buffer_detach
• MPI_Cancel
• MPI_Cart_coords
• MPI_Cart_create
• MPI_Cart_get
• MPI_Cart_map
• MPI_Cart_rank
• MPI_Cart_shift
• MPI_Cart_sub
• MPI_Cartdim_get
• MPI_Close_port
• MPI_Comm_accept
• MPI_Comm_c2f
• MPI_Comm_call_errhandler
• MPI_Comm_compare
• MPI_Comm_connect
• MPI_Comm_create
• MPI_Comm_create_errhandler
• MPI_Comm_create_group
• MPI_Comm_create_keyval
• MPI_Comm_delete_attr
• MPI_Comm_disconnect
• MPI_Comm_dup
• MPI_Comm_dup_with_info
• MPI_Comm_f2c
• MPI_Comm_free
• MPI_Comm_free_keyval
• MPI_Comm_get_attr
• MPI_Comm_get_errhandler
• MPI_Comm_get_info
• MPI_Comm_get_name
• MPI_Comm_get_parent
• MPI_Comm_group
• MPI_Comm_idup
• MPI_Comm_join
• MPI_Comm_rank
• MPI_Comm_remote_group
• MPI_Comm_remote_size
• MPI_Comm_set_attr
• MPI_Comm_set_errhandler
• MPI_Comm_set_info
• MPI_Comm_set_name
• MPI_Comm_size
• MPI_Comm_spawn
• MPI_Comm_spawn_multiple
• MPI_Comm_split
• MPI_Comm_split_type
• MPI_Comm_test_inter
• MPI_Compare_and_swap
• MPI_Dims_create
• MPI_Dist_graph_create
• MPI_Dist_graph_create_adjacent
• MPI_Dist_graph_neighbors
• MPI_Dist_graph_neighbors_count
• MPI_Errhandler_create
• MPI_Errhandler_free
• MPI_Errhandler_get
• MPI_Errhandler_set
• MPI_Error_class
• MPI_Error_string
• MPI_Exscan
• MPI_Fetch_and_op
• MPI_File_c2f
• MPI_File_call_errhandler
• MPI_File_close
• MPI_File_create_errhandler
• MPI_File_delete
• MPI_File_f2c
• MPI_File_get_amode
• MPI_File_get_atomicity
• MPI_File_get_byte_offset
• MPI_File_get_errhandler
• MPI_File_get_group
• MPI_File_get_info
• MPI_File_get_position
• MPI_File_get_position_shared
• MPI_File_get_size
• MPI_File_get_type_extent
• MPI_File_get_view
• MPI_File_iread
• MPI_File_iread_all
• MPI_File_iread_at
• MPI_File_iread_at_all
• MPI_File_iread_shared
• MPI_File_iwrite
• MPI_File_iwrite_all
• MPI_File_iwrite_at
• MPI_File_iwrite_at_all
• MPI_File_iwrite_shared
• MPI_File_open
• MPI_File_preallocate
• MPI_File_read
• MPI_File_read_all
• MPI_File_read_all_begin
• MPI_File_read_all_end
• MPI_File_read_at
• MPI_File_read_at_all
• MPI_File_read_at_all_begin
• MPI_File_read_at_all_end
• MPI_File_read_ordered
• MPI_File_read_ordered_begin
• MPI_File_read_ordered_end
• MPI_File_read_shared
• MPI_File_seek
• MPI_File_seek_shared
• MPI_File_set_atomicity
• MPI_File_set_errhandler
• MPI_File_set_info
• MPI_File_set_size
• MPI_File_set_view
• MPI_File_sync
• MPI_File_write
• MPI_File_write_all
• MPI_File_write_all_begin
• MPI_File_write_all_end
• MPI_File_write_at
• MPI_File_write_at_all
• MPI_File_write_at_all_begin
• MPI_File_write_at_all_end
• MPI_File_write_ordered
• MPI_File_write_ordered_begin
• MPI_File_write_ordered_end
• MPI_File_write_shared
• MPI_Finalize
• MPI_Finalized
• MPI_Free_mem
• MPI_Gather
• MPI_Gatherv
• MPI_Get
• MPI_Get_accumulate
• MPI_Get_address
• MPI_Get_count
• MPI_Get_elements
• MPI_Get_elements_x
• MPI_Get_library_version
• MPI_Get_processor_name
• MPI_Get_version
• MPI_Graph_create
• MPI_Graph_get
• MPI_Graph_map
• MPI_Graph_neighbors
• MPI_Graph_neighbors_count
• MPI_Graphdims_get
• MPI_Grequest_complete
• MPI_Grequest_start
• MPI_Group_c2f
• MPI_Group_compare
• MPI_Group_difference
• MPI_Group_excl
• MPI_Group_f2c
• MPI_Group_free
• MPI_Group_incl
• MPI_Group_intersection
• MPI_Group_range_excl
• MPI_Group_range_incl
• MPI_Group_rank
• MPI_Group_size
• MPI_Group_translate_ranks
• MPI_Group_union
• MPI_Iallgather
• MPI_Iallgatherv
• MPI_Iallreduce
• MPI_Ialltoall
• MPI_Ialltoallv
• MPI_Ialltoallw
• MPI_Ibarrier
• MPI_Ibcast
• MPI_Ibsend
• MPI_Iexscan
• MPI_Igather
• MPI_Igatherv
• MPI_Improbe
• MPI_Imrecv
• MPI_Ineighbor_allgather
• MPI_Ineighbor_allgatherv
• MPI_Ineighbor_alltoall
• MPI_Ineighbor_alltoallv
• MPI_Ineighbor_alltoallw
• MPI_Info_c2f
• MPI_Info_create
• MPI_Info_delete
• MPI_Info_dup
• MPI_Info_env
• MPI_Info_f2c
• MPI_Info_free
• MPI_Info_get
• MPI_Info_get_nkeys
• MPI_Info_get_nthkey
• MPI_Info_get_valuelen
• MPI_Info_set
• MPI_Init
• MPI_Init_thread
• MPI_Initialized
• MPI_Intercomm_create
• MPI_Intercomm_merge
• MPI_Iprobe
• MPI_Irecv
• MPI_Ireduce
• MPI_Ireduce_scatter
• MPI_Ireduce_scatter_block
• MPI_Irsend
• MPI_Is_thread_main
• MPI_Iscan
• MPI_Iscatter
• MPI_Iscatterv
• MPI_Isend
• MPI_Issend
• MPI_Keyval_create
• MPI_Keyval_free
• MPI_Lookup_name
• MPI_Message_c2f
• MPI_Message_f2c
• MPI_Mprobe
• MPI_Mrecv
• MPI_Neighbor_allgather
• MPI_Neighbor_allgatherv
• MPI_Neighbor_alltoall
• MPI_Neighbor_alltoallv
• MPI_Neighbor_alltoallw
• MPI_Op_c2f
• MPI_Op_commutative
• MPI_Op_create
• MPI_Op_f2c
• MPI_Op_free
• MPI_Open_port
• MPI_Pack
• MPI_Pack_external
• MPI_Pack_external_size
• MPI_Pack_size
• MPI_Pcontrol
• MPI_Probe
• MPI_Publish_name
• MPI_Put
• MPI_Query_thread
• MPI_Raccumulate
• MPI_Recv
• MPI_Recv_init
• MPI_Reduce
• MPI_Reduce_local
• MPI_Reduce_scatter
• MPI_Reduce_scatter_block
• MPI_Register_datarep
• MPI_Request_c2f
• MPI_Request_f2c
• MPI_Request_free
• MPI_Request_get_status
• MPI_Rget
• MPI_Rget_accumulate
• MPI_Rput
• MPI_Rsend
• MPI_Rsend_init
• MPI_Scan
• MPI_Scatter
• MPI_Scatterv
• MPI_Send
• MPI_Send_init
• MPI_Sendrecv
• MPI_Sendrecv_replace
• MPI_Sizeof
• MPI_Ssend
• MPI_Ssend_init
• MPI_Start
• MPI_Startall
• MPI_Status_c2f
• MPI_Status_f2c
• MPI_Status_set_cancelled
• MPI_Status_set_elements
• MPI_Status_set_elements_x
• MPI_T_category_changed
• MPI_T_category_get_categories
• MPI_T_category_get_cvars
• MPI_T_category_get_info
• MPI_T_category_get_num
• MPI_T_category_get_pvars
• MPI_T_cvar_get_info
• MPI_T_cvar_get_num
• MPI_T_cvar_handle_alloc
• MPI_T_cvar_handle_free
• MPI_T_cvar_read
• MPI_T_cvar_write
• MPI_T_enum_get_info
• MPI_T_enum_get_item
• MPI_T_finalize
• MPI_T_init_thread
• MPI_T_pvar_get_info
• MPI_T_pvar_get_num
• MPI_T_pvar_handle_alloc
• MPI_T_pvar_handle_free
• MPI_T_pvar_read
• MPI_T_pvar_readreset
• MPI_T_pvar_reset
• MPI_T_pvar_session_create
• MPI_T_pvar_session_free
• MPI_T_pvar_start
• MPI_T_pvar_stop
• MPI_T_pvar_write
• MPI_Test
• MPI_Test_cancelled
• MPI_Testall
• MPI_Testany
• MPI_Testsome
• MPI_Topo_test
• MPI_Type_c2f
• MPI_Type_commit
• MPI_Type_contiguous
• MPI_Type_create_darray
• MPI_Type_create_f90_complex
• MPI_Type_create_f90_integer
• MPI_Type_create_f90_real
• MPI_Type_create_hindexed
• MPI_Type_create_hindexed_block
• MPI_Type_create_hvector
• MPI_Type_create_indexed_block
• MPI_Type_create_keyval
• MPI_Type_create_resized
• MPI_Type_create_struct
• MPI_Type_create_subarray
• MPI_Type_delete_attr
• MPI_Type_dup
• MPI_Type_extent
• MPI_Type_f2c
• MPI_Type_free
• MPI_Type_free_keyval
• MPI_Type_get_attr
• MPI_Type_get_contents
• MPI_Type_get_envelope
• MPI_Type_get_extent
• MPI_Type_get_extent_x
• MPI_Type_get_name
• MPI_Type_get_true_extent
• MPI_Type_get_true_extent_x
• MPI_Type_hindexed
• MPI_Type_hvector
• MPI_Type_indexed
• MPI_Type_lb
• MPI_Type_match_size
• MPI_Type_set_attr
• MPI_Type_set_name
• MPI_Type_size
• MPI_Type_size_x
• MPI_Type_struct
• MPI_Type_ub
• MPI_Type_vector
• MPI_Unpack
• MPI_Unpack_external
• MPI_Unpublish_name
• MPI_Wait
• MPI_Waitall
• MPI_Waitany
• MPI_Waitsome
• MPI_Win_allocate
• MPI_Win_allocate_shared
• MPI_Win_attach
• MPI_Win_c2f
• MPI_Win_call_errhandler
• MPI_Win_complete
• MPI_Win_create
• MPI_Win_create_dynamic
• MPI_Win_create_errhandler
• MPI_Win_create_keyval
• MPI_Win_delete_attr
• MPI_Win_detach
• MPI_Win_f2c
• MPI_Win_fence
• MPI_Win_flush
• MPI_Win_flush_all
• MPI_Win_flush_local
• MPI_Win_flush_local_all
• MPI_Win_free
• MPI_Win_free_keyval
• MPI_Win_get_attr
• MPI_Win_get_errhandler
• MPI_Win_get_group
• MPI_Win_get_info
• MPI_Win_get_name
• MPI_Win_lock
• MPI_Win_lock_all
• MPI_Win_post
• MPI_Win_set_attr
• MPI_Win_set_errhandler
• MPI_Win_set_info
• MPI_Win_set_name
• MPI_Win_shared_query
• MPI_Win_start
• MPI_Win_sync
• MPI_Win_test
• MPI_Win_unlock
• MPI_Win_unlock_all
• MPI_Win_wait
• MPI_Wtick
• MPI_Wtime
• MPI_Allgather_init
• MPI_Allgatherv_init
• MPI_Allreduce_init
• MPI_Alltoall_init
• MPI_Alltoallv_init
• MPI_Alltoallw_init
• MPI_Barrier_init
• MPI_Bcast_init
• MPI_Comm_create_from_group
• MPI_Comm_idup_with_info
• MPI_Exscan_init
• MPI_Gather_init
• MPI_Gatherv_init
• MPI_Group_from_session_pset
• MPI_Info_create_env
• MPI_Info_get_string
• MPI_Intercomm_create_from_groups
• MPI_Isendrecv
• MPI_Isendrecv_replace
• MPI_Neighbor_allgather_init
• MPI_Neighbor_allgatherv_init
• MPI_Neighbor_alltoall_init
• MPI_Neighbor_alltoallv_init
• MPI_Neighbor_alltoallw_init
• MPI_Parrived
• MPI_Pready
• MPI_Pready_list
• MPI_Pready_range
• MPI_Precv_init
• MPI_Psend_init
• MPI_Reduce_init
• MPI_Reduce_scatter_block_init
• MPI_Reduce_scatter_init
• MPI_Scan_init
• MPI_Scatter_init
• MPI_Scatterv_init
• MPI_Session_call_errhandler
• MPI_Session_create_errhandler
• MPI_Session_c2f
• MPI_Session_f2c
• MPI_Session_finalize
• MPI_Session_get_errhandler
• MPI_Session_get_info
• MPI_Session_get_nth_pset
• MPI_Session_get_num_psets
• MPI_Session_get_pset_info
• MPI_Session_init
• MPI_Session_set_errhandler
• MPI_T_category_get_events
• MPI_T_category_get_index
• MPI_T_category_get_num_events
• MPI_T_cvar_get_index
• MPI_T_pvar_get_index
• MPI_T_event_callback_get_info
• MPI_T_event_callback_set_info
• MPI_T_event_copy
• MPI_T_event_get_num
• MPI_T_event_get_info
• MPI_T_event_get_index
• MPI_T_event_get_source
• MPI_T_event_get_timestamp
• MPI_T_event_handle_alloc
• MPI_T_event_handle_free
• MPI_T_event_handle_get_info
• MPI_T_event_handle_set_info
• MPI_T_event_read
• MPI_T_event_register_callback
• MPI_T_event_set_dropped_handler
• MPI_T_source_get_info
• MPI_T_source_get_num
• MPI_T_source_get_timestamp

Future Work

• See https://github.com/acdemiralp/mpi/projects/1