PRTE fails to link with hwloc 1.5.2 and 1.8.1

[jsquyres]

This was found in Cisco MTT: building Open MPI master with an external hwloc 1.5.2 and 1.8.1

Here's the external hwloc 1.5.2 stderr:

$ ./configure --prefix=/home/jsquyres/bogus --with-hwloc=/home/mpiteam/hwloc-1.5.2/install
...
$ make -j 32
...
hwloc/hwloc_base_util.c: In function ‘prte_hwloc_base_filter_cpus’:
hwloc/hwloc_base_util.c:280:15: warning: implicit declaration of function ‘hwloc_get_numanode_obj_by_os_index’; did you mean ‘hwloc_get_pu_obj_by_os_index’? [-Wimplicit-function-declaration]
         obj = hwloc_get_numanode_obj_by_os_index(topo, w);
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               hwloc_get_pu_obj_by_os_index
hwloc/hwloc_base_util.c:280:13: warning: assignment to ‘hwloc_obj_t’ {aka ‘struct hwloc_obj *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
         obj = hwloc_get_numanode_obj_by_os_index(topo, w);
             ^
  CCLD     libprrte.la
make[4]: Leaving directory `/home/jsquyres/git/o3/3rd-party/prrte/src'
Making all in tools/prted
make[4]: Entering directory `/home/jsquyres/git/o3/3rd-party/prrte/src/tools/prted'
  CC       prted.o
  CCLD     prted
../../../src/.libs/libprrte.so: undefined reference to `hwloc_get_numanode_obj_by_os_index'
collect2: error: ld returned 1 exit status
make[4]: *** [prted] Error 1

Note that the declaration problems are just warnings; there's a linker error that is the cause for make to fail.

Using external hwloc 1.8.1, the compile warnings / link error is the same as it is for external hwloc 1.5.2.

If the minimum hwloc version is simply higher than 1.8.1 these days, just let me know and I'll update Cisco's MTT (as long as the v5.0.x RMs are ok with increasing the minimum hwloc version -- that's a different conversation).

It would be nice to fail earlier (e.g., in configure) if we're using an hwloc that is unsupported, though.

Per discussion on the 11 Jan 2022 webex, marked this as "blocker".

[bgoglin]

hwloc_get_numanode_obj_by_os_index() was added in 1.10. It's easy to reimplement it when hwloc is old by adding something like:

#if HWLOC_API_VERSION < 0x10a00
hwloc_obj_t hwloc_get_numanode_obj_by_os_index(hwloc_topology_t topology, unsigned os_index)
{
  hwloc_obj_t obj = NULL;
  while ((obj = hwloc_get_next_obj_by_type(topology, HWLOC_OBJ_NUMANODE, obj)) != NULL)
    if (obj->os_index == os_index)
      return obj;
  return NULL;
}
#endif

[rhc54]

Hmmm...I have notes from a prior discussion that we were no longer going back that far in HWLOC support - I show 1.11 as the furthest. However, I have no problem adding the proposed patch.

[bwbarrett]

@rhc54 I couldn't remember what we decided while we were talking about this issue on the OMPI call, so that was the first thing I was going to check. There's a bug here regardless, because either 1) we should support hwloc 1.5.2 or 2) the configure test should abort with a message that says it can't find a new enough configure. I'm guessing the code for (2) is broken, which is why I had Jeff assign the issue to me...

[rhc54]

Borked configure could be right - does OMPI want to support 1.5.2? I don't really care - like I said, this patch is trivial, though IIRC there are some other impacts (we compile, but may see odd things in terms of support).

Thx for taking a look at the configure question. I'll post a PR with the patch from Brice as a backup plan.

[awlauria]

I remember the line we drew in the sand for v5.0.x being hwloc v1.11.

[rhc54]

Yeah, that matches my memory - on the other hand, PRRTE is used by other orgs, and I have no idea how far back they go. Since the fix is so simple, I'll just add it in.

[rhc54]

Done - someone should update the OMPI master submodule pointers to pick it up.

[jsquyres]

Ok. I've updated Cisco's MTT for master / v5.0.x to only test with external hwloc 1.11.1.

Let's leave this issue open until the PRTE submodule pointer is updated here in OMPI.

[rhc54]

Should you update OMPI's configure to reject earlier versions of hwloc?