Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mpi4py #454

Merged
merged 4 commits into from
Feb 4, 2019
Merged

mpi4py #454

merged 4 commits into from
Feb 4, 2019

Conversation

ax3l
Copy link
Member

@ax3l ax3l commented Jan 27, 2019

Add support for MPI-parallel I/O in python3 bindings.

Close #99

To Do

  • CMake Logic
  • min. versions in docs (should work with any mpi4py version, at least for > 1.0, from what the impls for the comm type look; but mpi4py.run options are new in 2.1+)
  • Examples
  • Unit Tests
  • extras in setup.py

//! https://bitbucket.org/mpi4py/mpi4py/src/3.0.0/demo/wrap-c/helloworld.c
// if( import_mpi4py() < 0 ) { here be dragons }

MPI_Comm *mpiCommPtr = PyMPIComm_Get(comm.ptr());

This comment was marked as resolved.

//! @todo perform mpi4py import test?
//! required C-API init?
//! https://bitbucket.org/mpi4py/mpi4py/src/3.0.0/demo/wrap-c/helloworld.c
// if( import_mpi4py() < 0 ) { here be dragons }
Copy link
Member Author

@ax3l ax3l Jan 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to avoid mpi4py/mpi4py.h include:
Function generated during build. This just seems to import and perform tests on functions and type signatures for binary compatibility (e.g. the usual: MPI during build time must be MPI at runtime).
Afterwards the imported test module is unloaded again. We can probably skip it and check e.g. versions with pybind11's import method.

@ax3l ax3l force-pushed the topic-mpi4py branch 5 times, most recently from 8d9137a to 5bb349c Compare January 29, 2019 00:07
CMakeLists.txt Outdated
@@ -276,6 +276,11 @@ if(openPMD_HAVE_PYTHON)
INTERFACE pybind11::pybind11)
endif()

# TODO: Check if mpi4py is available
# ref: https://github.com/ornladios/ADIOS2/blob/v2.3.0/cmake/FindPythonModule.cmake
# find_package(PythonModule REQUIRED COMPONENTS mpi4py mpi4py/mpi4py.h)

This comment was marked as outdated.

"(Mismatched MPI at compile vs. runtime?)");
}

return new Series(filepath, at, *mpiCommPtr); //! @todo do we already clone the communicator?

This comment was marked as outdated.

This comment was marked as outdated.

@ax3l ax3l force-pushed the topic-mpi4py branch 2 times, most recently from 4b42ad4 to b1b0905 Compare January 30, 2019 14:48
${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/${examplename}.py
WORKING_DIRECTORY ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}
)
endif()

This comment was marked as resolved.

@ax3l ax3l mentioned this pull request Jan 30, 2019
@ax3l ax3l force-pushed the topic-mpi4py branch 6 times, most recently from b72c2f6 to 4a97a1f Compare January 31, 2019 10:05
@ax3l ax3l force-pushed the topic-mpi4py branch 2 times, most recently from 9414398 to e3dc89c Compare January 31, 2019 15:07
@ax3l ax3l changed the title [Draft] mpi4py [WIP] mpi4py Jan 31, 2019
ax3l added 2 commits February 4, 2019 01:31
Add bindings for passing an mpi4py MPI communicator to
our Python `Series` object.
Also prepare CMake logic for examples.
translate the parallel C++ examples to Python
Document required versions and its support.
// pyMPI: ... (todo)
py::str const comm_pystr = py::repr(comm);
std::string const comm_str = comm_pystr.cast<std::string>();
if( comm_str.substr(0, 12) != std::string("<mpi4py.MPI.") )
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably only match mpi4py Communicators (and derived/same-layouted objects)

Define an "openPMD-api[mpi]" optional variant in setup.py,
which will pull mpi4py.
@ax3l ax3l changed the title [WIP] mpi4py mpi4py Feb 4, 2019
@ax3l ax3l merged commit 0a423f7 into openPMD:dev Feb 4, 2019
@ax3l ax3l deleted the topic-mpi4py branch February 4, 2019 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant