Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for parallel IO with mpi4py #717

Closed
jswhit opened this issue Sep 25, 2017 · 6 comments
Closed

Add support for parallel IO with mpi4py #717

jswhit opened this issue Sep 25, 2017 · 6 comments

Comments

@jswhit
Copy link
Collaborator

jswhit commented Sep 25, 2017

add parallel IO capabilities. netcdf-c and hdf5 must be compiled with MPI support, and mpi4py must be installed. To open a file for parallel access, use parallel=True in Dataset.__init__ and optionally pass the mpi4py Comm instance using the comm kwarg and the mpi4py Info instance using the info kwarg. If neither are specified, MPI_COMM_WORLD and MPI_INFO_NULL are used. IO can be toggled between collective and independent using Variable.set_collective. See examples/mpi_example.py and pull request #716.

@jswhit
Copy link
Collaborator Author

jswhit commented Sep 25, 2017

There are no tests yet, but for some reason tst_diskless.py fails when the an mpi-enabled netcdf-c library is linked.

@jswhit
Copy link
Collaborator Author

jswhit commented Sep 27, 2017

Travis now runs examples/mpi_example.py. Failures are allowed for now while this support is experimental.

@jswhit
Copy link
Collaborator Author

jswhit commented Sep 28, 2017

Seems ready to go, will merge tomorrow if there are no comments.

@jswhit
Copy link
Collaborator Author

jswhit commented Sep 28, 2017

pull request #716 merged

@ArchangeGabriel
Copy link
Contributor

@jswhit Where you able to pin/fix the tst_diskless.py failure? I had reported it a while ago #669 so that some tests had already been done around it.

@jswhit
Copy link
Collaborator Author

jswhit commented Oct 21, 2017

No - someone should probably create at netcdf-c issue for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants