-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
APT whitelist request for mpich-bin #406
Comments
Ran tests and found setuid bits by purely textual search. Further analysis is required. If these are found to be benign, add: libmpich-dev libmpich12 libmpich2-dev mpich mpich-doc mpich2 mpich2-doc libmpl-dev libmpl1 libopa-dev libopa1 See https://travis-ci.org/travis-ci/apt-whitelist-checker/builds/72545568. |
@BanzaiMan - I'm wondering if you could provide update on the progress of this issue. I'm working on a project that needs |
bump? |
Any update on this whitelist? |
I'd also really like to get mpich2. libmpich2-3, and libmpich2-dev whitelisted. |
I submited a PR for mpich2 and libmpich2-dev (that should pull libmpich2-3 also) |
@BanzaiMan those occurences of setuid are bening, two are just comments, and the other is in mpd which should not really be used (one should use hydra instead). |
I would really like to be able to install MPICH, MVAPICH and OpenMPI from container based builds. Any status update, @BanzaiMan ? |
+1 We need this too. Are there still unresolved issues with these packages? |
@BanzaiMan I saw that you have looked at this several times in other issues. Any progress? |
There are three hits for Line 442Offending text
AnalysisThis code is called in the MPD process manager, which is the old processor manager for MPICH. It is not used by default and has been deprecated for a long time (https://wiki.mpich.org/mpich/index.php/Frequently_Asked_Questions#Q:_I_don.27t_like_.3CWHATEVER.3E_about_mpd.2C_or_I.27m_having_a_problem_with_mpdboot.2C_can_you_fix_it.3F). I don't know if the MPICH binary includes all the process managers or not, but at least the default Line 456Offending text
AnalysisFirst, this is in a comment in the source code, hence is inconsequently to the binary installation. Second, the offending source code is part of hwloc, which is already whitelisted (https://github.com/travis-ci/apt-package-whitelist/blob/master/ubuntu-precise#L1551). Furthermore, hwloc is integrated into Open-MPI, which has already been whitelisted (https://github.com/travis-ci/apt-package-whitelist/blob/master/ubuntu-precise#L6480). Third, this code is for Solaris. Unless Travis CI supports the Solaris operating system, it is not relevant. Line 468Offending text
AnalysisThis text is in the
I contend that all of these are benign. The only one that is even remotely relevant is in
|
For what it's worth, MPICH is trivial to build from source as part of Travis, and the time to build is not onerous. You can even add
|
Before we think about this any harder, let us first get the right package, meaning the binary one rather than the source one:
(that's from https://travis-ci.org/travis-ci/apt-whitelist-checker/builds/72545568). |
@jeffhammond @BanzaiMan |
@zbeekman The point of disabling Fortran, in the context of @jeffhammond's comment, was to speed up builds where project maintainers are forced to build MPICH themselves, as opposed to installing a package. It is certainly valid to disable Fortran wrappers when building MPICH for your project, if it does not require Fortran. If we are talking about installing packages, then it should certainly include Fortran wrappers so it is useful for everyone. |
@justusc ah fair point. I read it too quickly, trying to catch up after a week of vacation. |
Just remember that Fortran modules may not be compatible across versions of Note also that I've written tens of thousands of lines of Fortran in my |
@jeffhammond yup, I know you're a Fortran guru. As I said, I read it too quickly and just wanted to make sure that any whitelisted apt package would include fortran bindings. Sorry again for miss-reading what you were saying. |
@jeffhammond also, 👍 for using higher level libraries rather than interfacing directly. 😄 https://GitHub.com/sourceryinstitute/opencoarrays ++ |
@BanzaiMan Is there any progress on this? This has been open for around half a year now and from what ascertain in the thread, the "Further analysis is required" has been answered via Jeff. Is there anything else blocking this? |
@BanzaiMan - bump! If we could get some feedback here, it would be much appreciated. |
@jeffhammond I add the code below to my
The changes compared to your code are:
With these, it seems that I can compile with mpicc under travis. |
As @jeffhammond said, you can install mpich from source, and you can also cache the build directory, so that it doesn't have to be build every single time, as described in this blog post: https://d-meiser.github.io/2016/01/10/mpi-travis.html For my projects, I just install openmpi, which is whitelisted already, and that works to test my codes in parallel on Travis. But it would be nice to white list mpich as well. |
The main reason I prefer to run the automatic tests with MPICH instead of with OpenMPI is its strongly typed validation layer that automatically works with clang. That is, if you try to pass a There is a clang-tidy check in LLVM 3.9 that should do this independently of the MPI implementation used, but it is new, and it doesn't work through typedefs and such (there are actually two checks already, and more might come). This might turn out to be a solution in the future for those using OpenMPI, but integrating clang-tidy in an already existing project can take a lot of effort (if one wants to do it right). |
We do not need to debate Open-MPI vs MPICH here. What we need is for Travis to whitelist MPICH, not in the least because they have whitelisted packages that depend on it. It is absurd that nothing is happening after 15 months and intense interest from users. I performed the detailed analysis requested 9 months ago (#406 (comment)), but it appears that no one will act on it. |
Ping @BanzaiMan |
FWIW I've contacted travis support multiple times saying that I would love to upgrade to paid/pro travis version if that would mean that they would provide "basic" C/C++ support, where for basic I meant "no months long test breakage due to non-white-listed new clang/gcc versions, and quickly whitelisting new releases of the fundamental C and C++ libraries that I needed (MPICH, parallel-netcdf, hdf5, ...)" (I think these are reasonable requirements). Their answer was that they were not interested. I would hope they would change their mind. Maybe they'll do so if they see enough of a market in C and C++ testing. |
Is there any progress in this issue, or will it be closed as At the moment I'm using the OS X infrastructure for testing with MPICH, but I think this is overkill, as there has to be an entire VM booted for every run, especially because Travis OS X builds are heavily overcommitted these days. |
Yes it's absurd that this hasn't been accepted yet, especially since Jeff
Hammond did all of the required set uid set gid analysis.
…On Wed, Jan 25, 2017 at 3:11 PM Alexander Haase ***@***.***> wrote:
Is there any progress in this issue, or will it be closed as wontfix?
At the moment I'm using the OS X infrastructure for testing with MPICH,
but I think this is overkill, as there has to be an entire VM booted for
every run, especially because Travis OS X builds are heavily overcommitted
these days.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#406 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAREPH4k4IGk7ai-AgTEmm_gkI5GQTKLks5rV6yOgaJpZM4FffJp>
.
|
@BanzaiMan any updates here? |
This commit aims to resolve problems for the upgrade of the default macOS image on Travis CI: https://goo.gl/H7hBsg The upgrade brings - "brew install mpich" fails by default (travis-ci/travis-ci#8826), - The fixed version valgrind-3.11.0 doesn't work (Sierra is not supported). Open MPI is "fragile" with Valgrind and gcov, while MPICH is not in the APT whitelist (travis-ci/apt-package-safelist#406), so we used to use brewed MPICH on macOS (but the latest Valgrind gives false positives). Now change the strategy: manually build and cache MPICH on Ubuntu and use it with Valgrind and gcov. Test jobs on macOS are reduced to only "osx-bin-release", which indeed checks FORM and TFORM. It also contains refactoring of check.rb: - Show more readable error messages, - Add a RuboCop configuration (in my taste).
38 months is enough time to create four human beings (sequentially). Honestly, how is this ticket still unresolved? |
Packages: mpich-bin mpich-mpd-bin mpich-shmem-bin libmpich1.0-dev libmpich-mpd1.0-dev libmpich-shmem1.0-dev libmpich1.0gf libmpich-mpd1.0gf libmpich-shmem1.0gf mpi-doc mpe-source
This is an automated comment. Ran tests and found setuid bits by purely textual search. Further analysis is required. If these are found to be benign, examine http://github.com/travis-ci/apt-package-whitelist/compare/test-apt-package-whitelist-406 and its PR. Packages found: mpich-bin mpich-mpd-bin mpich-shmem-bin libmpich1.0-dev libmpich-mpd1.0-dev libmpich-shmem1.0-dev libmpich1.0gf libmpich-mpd1.0gf libmpich-shmem1.0gf mpi-doc mpe-source See https://travis-ci.org/travis-ci/apt-whitelist-checker/builds/440490148 for details. |
This replaces travis-ci/travis-ci#4339.
The original text by @QuLogic follows
#4164 conveniently added not only
libhdf5-openmpi-dev
, but alsolibhdf5-mpich-dev
andlibhdf5-lam-dev
. The trouble is thatmpich-bin
andlam-runtime
are not whitelisted, so you can't run any MPI-compiled binaries.The text was updated successfully, but these errors were encountered: