Skip to content

Commit

Permalink
btl/base: add a subsystem for supporting active-message "RDMA"
Browse files Browse the repository at this point in the history
This commit adds a new subsystem to the btl base: active-message
RDMA. The subsystem can be activated on any non-RDMA or partial-
RDMA btl (for example tcp) by calling:

```c
mca_btl_base_am_rdma_init(mca_btl_base_module_t *btl).
```

Once called the btl module supplied with be augmented to have
btl_get, btl_put, btl_fop, and btl_cswap functions. Flags will
be set to indicate that the BTL is using active-message
versions of these functions.

Calling this function on a btl will register a progress
function for handling retries of intiator or target-side
operations. There may be some overhead once the progress
function is registered.

Active-message "RDMA" support is not activated on any BTL
module by default.

Signed-off-by: Nathan Hjelm <hjelmn@google.com>
  • Loading branch information
hjelmn committed Mar 2, 2021
1 parent e813ad3 commit 0d8140c
Show file tree
Hide file tree
Showing 8 changed files with 1,139 additions and 8 deletions.
8 changes: 6 additions & 2 deletions opal/mca/btl/base/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2021 Triad National Security, LLC. All rights reserved.
# Copyright (c) 2021 Google, LLC. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
Expand All @@ -20,10 +22,12 @@ dist_opaldata_DATA = base/help-mpi-btl-base.txt

headers += \
base/base.h \
base/btl_base_error.h
base/btl_base_error.h \
base/btl_base_am_rdma.h

libmca_btl_la_SOURCES += \
base/btl_base_frame.c \
base/btl_base_error.c \
base/btl_base_select.c \
base/btl_base_mca.c
base/btl_base_mca.c \
base/btl_base_am_rdma.c
5 changes: 5 additions & 0 deletions opal/mca/btl/base/base.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
* All rights reserved.
* Copyright (c) 2006 Sun Microsystems, Inc. All rights reserved.
* Copyright (c) 2008 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2020 Google, LLC. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
Expand All @@ -28,6 +29,10 @@
#include "opal/mca/mca.h"
#include "opal/mca/btl/btl.h"

#define MCA_BTL_BASE_TAG_RDMA MCA_BTL_TAG_BTL_BASE
#define MCA_BTL_BASE_TAG_ATOMIC (MCA_BTL_TAG_BTL_BASE + 1)
#define MCA_BTL_BASE_TAG_RDMA_RESP (MCA_BTL_TAG_BTL_BASE + 2)

BEGIN_C_DECLS

struct mca_btl_base_selected_module_t {
Expand Down
Loading

0 comments on commit 0d8140c

Please sign in to comment.