RPM build options --define 'install_in_opt 1' and --define 'install_modulefile 1' don't interact properly

[jschwartz-cray]

Background information

Trying to build RPMs from the SRPM which get installed properly under /opt with an env modulefile in /etc/modulefiles.

What version of Open MPI are you using? (e.g., v1.10.3, v2.1.0, git branch name and hash, etc.)

v4.0.1

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Trying to build RPMs

Please describe the system on which you are running

• Operating system/version: CentOS Linux release 7.5.1804 (Core)
• Computer hardware: Docker
• Network type: N/A

---

Details of the problem

rpmbuild \
    --define 'install_in_opt 1' \
    --define 'install_modulefile 1' \
    --rebuild openmpi-4.0.1-1.src.rpm

Currently when these options are combined the modulefile gets installed under:

/opt/openmpi/4.0.1/share/openmpi/modulefiles

rather than the desired location of:

/etc/modulefiles

which defeats the purpose of installing in /opt. Generally people who are installing in /opt are doing so because they have many different versions installed at the same time and they want to be able to switch between them using module purge/load/unload. By putting the modulefile under /opt as well it adds a redundant post-install step to add the path to MODULEPATH.

When --define 'install_in_opt 1' is set, the spec file should not change %{_sysconfdir}.

[jsquyres]

Sorry for taking forever to get to this. Thanks for bringing this to our attention.

FWIW, although install_in_opt does redefine the modulefile installation location, it doesn't do it via %{_sysconfdir}.

That being said, the default modulefile location is /usr/share/Modules/modulefiles because -- at least back when this specfile was written -- there was no common, cross-distro location where modulefiles were supposed to be written. I don't remember exactly how that default location was picked, but it was probably where at least one distro (or HPC packager?) put them. Is `/etc/modulefiles/ standard across lots of distros / packagers these days?

Note that you can --define modulefile_path WHATEVER to set the installation path to be whatever you want. Unfortunately, it looks like this gets overwritten if install_in_opt is set, though.

Would you consider it better behavior to not overwrite the modulefile_path if a) install_in_opt==1 and b) modulefile_path was set to something by the user?

[jschwartz-cray]

I have no qualms with the default modulefile location, and the /usr/share/... location is probably the more modern one:

echo $MODULEPATH
/usr/share/Modules/modulefiles:/etc/modulefiles

The more important thing is that there's a sensible way to override it (that does not itself get overwritten).

I agree that your proposal is better than the existing behavior. My only quibble would be that I don't think it's appropriate for the modulefile to ever get installed under /opt unless someone explicitly passes a --define modulefile_path /opt/WHATEVER. The modulefiles are already organized by versions with the ability to designate a default. I don't think it's necessary for install_in_opt=1 to change the modulefile install location under any circumstances.

If someone really wants it in /opt they could use modulefile_path themselves to put it there. That is problematic though because the modulefile is itself the place where you would typically modify something like MODULEPATH to add the /opt location; this is a chicken and egg problem.

[jschwartz-cray]

Just an update on this, I believe this part of the openmpi.spec is wrong as well:

# If we're installing the modulefile, get that, too
%if %{install_modulefile}
%{modulefile_path}
%endif

I believe this should be changed to:

# If we're installing the modulefile, get that, too
%if %{install_modulefile}
%{modulefile_path}/%{modulefile_subdir}/%{modulefile_name}
%endif

or maybe:

# If we're installing the modulefile, get that, too
%if %{install_modulefile}
%{modulefile_path}/%{modulefile_subdir}
%endif

but we probably don't want it trying to claim ownership of and/or removing the default %{modulefile_path} of /usr/share/Modules/modulefiles.

EDIT: I had an incorrect comment about the %dir directive in here previously.

[jsquyres]

The more important thing is that there's a sensible way to override it (that does not itself get overwritten).

Agreed.

I agree that your proposal is better than the existing behavior.

Ok.

My only quibble would be that I don't think it's appropriate for the modulefile to ever get installed under /opt unless someone explicitly passes a --define modulefile_path /opt/WHATEVER.

Hmm. Not sure I agree with that.

FWIW, I believe that the original intent for the current scheme was multifold:

1. If someone specifies install_in_opt==1, then Open MPI should be entirely contained within /opt.
2. We cannot assume the directory structure of the end user's modulefiles setup. Hence, the end user can sym/hard link the modulefile that we provide into whatever directory structure they want.

That being said, allowing the user to override modulefile_path would seem to allow the 2nd option without needing an additional sym/hard link / post-install step.

The modulefiles are already organized by versions with the ability to designate a default. I don't think it's necessary for install_in_opt=1 to change the modulefile install location under any circumstances.

If someone really wants it in /opt they could use modulefile_path themselves to put it there. That is problematic though because the modulefile is itself the place where you would typically modify something like MODULEPATH to add the /opt location; this is a chicken and egg problem.

I agree that this is a chicken-n-egg problem -- we can make arguments for both ways. How about taking at least a definitive step forward in allowing the user to define modulefile_path to get the resulting modulefile wherever they want?

I just posted PR #7264 -- let's iterate over there.

[jsquyres]

but we probably don't want it trying to claim ownership of and/or removing the default %{modulefile_path} of /usr/share/Modules/modulefiles.

I'm of two minds about this.

1. On the one hand, we should just remove that one file.
2. On the other hand, if we're the only file in that tree / that tree was created just for this file, we should remove that tree, too.

I did a few tests:

1. Listed just %{modulefile_path} in %files
   • In this case, the directory is created (if necessary) upon installation
   • Upon RPM removal, the entire {modulefile_path} tree removed (unless other files are in that tree that were not put there by installing the RPM, of course)
2. Listed just %{modulefile_path}/%{modulefile_subdir}/%{modulefile_name} in %files
   • In this case, the directory is created (if necessary) upon installation
   • Upon RPM removal, the entire directory tree remains -- only the file is removed
3. Listed %{modulefile_path}/%{modulefile_subdir}/%{modulefile_name} and %dir {modulefile_path} in %files
   • In this case, the directory is created (if necessary) upon installation
   • Upon RPM removal, the file is removed, but the directory tree remains (!!)

The third case surprised me a bit. Shrug.

Regardless, I believe that the intent for the Open MPI spec file here is to remove the entire modulefile path tree if we were the only file that lived in that tree (vs. rpm -e still leaving some empty directories around). That's why we list {modulefile_path} in %files.

Make sense?