Merge #3234 #3236 #3238

3234: Fix mpiio with stdlibc++ range checking r=fweik a=mkuron

Fixes #3230. Reported by @junghans.

When mpiio was used but no bonds were present, we would still try to copy zero bonds from a zero-length vector. This triggered an assertion when stdlibc++ range checking was enabled.

Please tag for cherry-picking into 4.1.1.

3236: ESS2019 installation guide updates r=KaiSzuttor a=mkuron

Lessons learned today:

- We require MPI 3 because we depend on const-correctness in a few places. That means that OpenMPI 1.6.5 and lower are not supported anymore.
- Installing the ROCm driver breaks access to /dev/kfd, causing hwloc initialization during `mpiexec` to hang. Rebooting helps.
- Add matplotlib, ipython and jupyter to the Mac install guide.
- Homebrew now defaults to Python 3, requires manually enabling cython, and it's unclear whether the hdf5 package still supports MPI (Homebrew/homebrew-core#26974)
- Anaconda (~/anaconda[23]) and python.org packages (/Library/Python and /usr/local/bin) are also sources of conflict

Please tag for the 4.1.1 release

3238: maintainer: Escape module python in wrapper script r=jngrad a=fweik

Fixes #3237.

Description of changes:
 - Added quotes around the module path in python wrapper script.


Co-authored-by: Michael Kuron <mkuron@users.noreply.github.com>
Co-authored-by: Michael Kuron <mkuron@icp.uni-stuttgart.de>
Co-authored-by: Kai Szuttor <kai@icp.uni-stuttgart.de>
Co-authored-by: Florian Weik <fweik@icp.uni-stuttgart.de>

CMakeLists.txt

@@ -336,7 +336,7 @@ endif(WITH_VALGRIND_INSTRUMENTATION)
 # MPI
 #
 
-find_package(MPI REQUIRED)
+find_package(MPI 3.0 REQUIRED)
 
 # CMake < 3.9
 if(NOT TARGET MPI::MPI_CXX)

doc/sphinx/installation.rst

@@ -66,15 +66,15 @@ Cython
 .. _Installing Requirements on Ubuntu Linux:
 
 Installing Requirements on Ubuntu Linux
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To make |es| run on 18.04 LTS, its dependencies can be installed with:
 
 .. code-block:: bash
 
     sudo apt install build-essential cmake cython3 python3-numpy \
-    libboost-all-dev openmpi-common fftw3-dev libhdf5-dev libhdf5-openmpi-dev \
-    doxygen python3-opengl python3-sphinx python3-pip libgsl-dev
+      libboost-all-dev openmpi-common fftw3-dev libhdf5-dev libhdf5-openmpi-dev \
+      python3-opengl libgsl-dev
 
 
 Optionally the ccmake utility can be installed for easier configuration:
@@ -88,7 +88,8 @@ are required:
 
 .. code-block:: bash
 
-    pip3 install --upgrade jupyter scipy matplotlib sphinxcontrib-bibtex numpydoc
+    sudo apt install python3-matplotlib python3-scipy \
+      ipython3 jupyter-notebook
 
 If your computer has an Nvidia graphics card, you should also download and install the
 CUDA SDK to make use of GPU computation:
@@ -114,68 +115,98 @@ ROCm SDK to make use of GPU computation:
     sudo apt update
     sudo apt install libnuma-dev rocm-dkms rocblas rocfft rocrand rocthrust
 
+After installing the ROCm SDK, please reboot your computer.
+
 .. _Installing Requirements on Mac OS X:
 
 Installing Requirements on Mac OS X
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To make |es| run on Mac OS X 10.9 or higher, its dependencies can be
-installed using MacPorts. First, download the installer package
-appropriate for your Mac OS X version from
-https://www.macports.org/install.php and install it. Then, run the
+Preparation
+"""""""""""
+
+To make |es| run on Mac OS X 10.9 or higher, you need to install its
+dependencies. There are two possibilities for this, MacPorts and Homebrew.
+We recommend MacPorts, but if you already have Homebrew installed, you can use
+that too. To check whether you already have one or the other installed, run the
 following commands:
 
 .. code-block:: bash
 
-    sudo xcode-select --install
-    sudo xcodebuild -license accept
-    sudo port selfupdate
-    sudo port install cmake python37 py37-cython py37-numpy \
-      openmpi-default fftw-3 +openmpi boost +openmpi +python37 \
-      doxygen py37-opengl py37-sphinx py37-pip gsl hdf5 +openmpi
-    sudo port select --set cython cython37
-    sudo port select --set python3 python37
-    sudo port select --set pip pip37
-    sudo port select --set mpi openmpi-mp
+    test -e /opt/local/bin/port && echo "MacPorts is installed"
+    test -e /usr/local/bin/brew && echo "Homebrew is installed"
 
-Alternatively, you can use Homebrew.
+If both are installed, you need to remove one of the two. To do that, run one
+of the following two commands:
+
+.. code-block:: bash
+
+    sudo port -f uninstall installed && rm -r /opt/local
+    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
+
+If Homebrew is already installed, you should resolve any problems reported by
+the command
+
+.. code-block:: bash
+
+    brew doctor
+
+If Anaconda Python or the Python from www.python.org are installed, you
+will likely not be able to run |es|. Therefore, please uninstall them
+using the following commands:
+
+.. code-block:: bash
+
+    sudo rm -r ~/anaconda[23]
+    sudo rm -r /Library/Python
+
+If you want to install MacPorts, download the installer package
+appropriate for your Mac OS X version from
+https://www.macports.org/install.php and install it.
+
+If you want to install Homebrew, use the following commands.
 
 .. code-block:: bash
 
     sudo xcode-select --install
     sudo xcodebuild -license accept
     /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-    brew install cmake python@3 cython boost boost-mpi fftw \
-      doxygen gsl
-    brew install hdf5 --with-mpi
-    brew install numpy --without-python@2
-    ln -s /usr/local/bin/python2 /usr/local/bin/python
-    pip install --user PyOpenGL
 
-Note: If both MacPorts and Homebrew are installed, you will not be able to
-run |es|. Therefore, if you have both installed, please uninstall one
-or the other by running one of the following two commands:
+Installing Packages using MacPorts
+""""""""""""""""""""""""""""""""""
+
+Run the following commands:
 
 .. code-block:: bash
 
-    sudo port -f uninstall installed && rm -r /opt/local
-    ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
+    sudo xcode-select --install
+    sudo xcodebuild -license accept
+    sudo port selfupdate
+    sudo port install cmake python37 py37-cython py37-numpy \
+      openmpi-default fftw-3 +openmpi boost +openmpi +python37 \
+      doxygen py37-opengl py37-sphinx gsl hdf5 +openmpi \
+      py37-matplotlib py37-ipython py37-jupyter
+    sudo port select --set cython cython37
+    sudo port select --set python3 python37
+    sudo port select --set mpi openmpi-mp
 
-If your Mac has an Nvidia graphics card, you should also download and install the
-CUDA SDK [6]_ to make use of GPU computation.
 
-.. _Installing python dependencies:
+Installing Packages using Homebrew
+""""""""""""""""""""""""""""""""""
 
-Installing python dependencies
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. code-block:: bash
 
-There are a few python packages needed to e.g. build the documentation.
-To install the required packages as a non-root user execute the following
-command in |es|'s source directory:
+    brew install cmake python cython boost boost-mpi fftw \
+      doxygen gsl numpy ipython jupyter
+    brew install hdf5
+    brew link --force cython
+    pip install PyOpenGL matplotlib
 
-.. code-block:: bash
+Installing CUDA
+"""""""""""""""
 
-    pip3 install -r requirements.txt --user --upgrade
+If your Mac has an Nvidia graphics card, you should also download and install the
+CUDA SDK [6]_ to make use of GPU computation.
 
 .. _Quick installation:
 
@@ -248,7 +279,7 @@ Configuring
 .. _myconfig.hpp\: Activating and deactivating features:
 
 :file:`myconfig.hpp`: Activating and deactivating features
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 |es| has a large number of features that can be compiled into the binary.
 However, it is not recommended to actually compile in all possible
@@ -317,7 +348,7 @@ and then in the Python interpreter:
 .. _Features:
 
 Features
-~~~~~~~~
+--------
 
 This chapter describes the features that can be activated in |es|. Even if
 possible, it is not recommended to activate all features, because this
@@ -532,15 +563,15 @@ Finally, there is a flag for debugging:
 
 
 Features marked as experimental
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Some of the above features are marked as EXPERIMENTAL. Activating these features can have unexpected side effects and some of them have known issues. If you activate any of these features, you should understand the corresponding source code and do extensive testing. Furthermore, it is necessary to define ``EXPERIMENTAL_FEATURES`` in :file:`myconfig.hpp`.
 
 
 
 .. _cmake:
 
 cmake
-~~~~~
+^^^^^
 
 In order to build the first step is to create a build directory in which
 cmake can be executed. In cmake, the *source directory* (that contains
@@ -572,7 +603,7 @@ Afterwards |es| can be run via calling :file:`./pypresso` from the command line.
 .. _ccmake:
 
 ccmake
-~~~~~~
+^^^^^^
 
 Optionally and for easier use, the curses interface to cmake can be used
 to configure |es| interactively.

src/core/io/mpiio/mpiio.cpp

@@ -448,7 +448,7 @@ void mpi_mpiio_common_read(const char *filename, unsigned fields) {
       int blen = boff[i + 1] - boff[i];
       auto &bl = local_particles[id[i]]->bl;
       bl.resize(blen);
-      std::copy_n(&bond[boff[i]], blen, bl.begin());
+      std::copy_n(bond.begin() + boff[i], blen, bl.begin());
     }
   }
 

src/python/pypresso.cmakein

@@ -8,9 +8,9 @@
 #
 
 if test -n "$PYTHONPATH"; then
-  PYTHONPATH=@PYTHON_DIR@:$PYTHONPATH
+  PYTHONPATH="@PYTHON_DIR@:$PYTHONPATH"
 else
-  PYTHONPATH=@PYTHON_DIR@
+  PYTHONPATH="@PYTHON_DIR@"
 fi
 export PYTHONPATH