Skip to content

Commit

Permalink
fix the openmpi build on gcc14/fedora 40
Browse files Browse the repository at this point in the history 
- add missing construct_event_strings.py
- add fix from upstream bug
  open-mpi/ompi#12169
  which then triggers the usage of construct_event_strings.py

fixes: #12

Signed-off-by: Mika Laitio <lamikr@gmail.com>
  • Loading branch information
@lamikr
lamikr committed May 29, 2024
1 parent 5ff2a33 commit b95d8b7
Show file tree
Hide file tree
Showing 3 changed files with 352 additions and 3 deletions.
6 changes: 3 additions & 3 deletions patches/rocm-6.1.1/openmpi/0001-disable-prrte-docs-generation.patch
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 47da35a3624cbdb205c1d2529f12faa5a868f54d Mon Sep 17 00:00:00 2001
From 03ced091cd830fe4d16ad1596228bc035e4b961c Mon Sep 17 00:00:00 2001
From: Mika Laitio <lamikr@gmail.com>
Date: Wed, 24 Jan 2024 14:53:02 -0800
Subject: [PATCH] disable prrte docs generation
Subject: [PATCH 1/3] disable prrte docs generation

Disable prrte doc building to avoid follwing error
if python sphinx is not installed available:
Expand Down Expand Up @@ -51,5 +51,5 @@ index a623100..f98c89e 100644
util \
$(MCA_prte_FRAMEWORKS_SUBDIRS) \
--
2.43.0
2.45.1

76 changes: 76 additions & 0 deletions patches/rocm-6.1.1/openmpi/0002-upstream-patch-Cast-a-few-parameters-when-translatin.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
From 9037de8d1cfb749d5bf8e8a2223de8352b9bf6b6 Mon Sep 17 00:00:00 2001
From: Mika Laitio <lamikr@gmail.com>
Date: Wed, 29 May 2024 12:09:47 -0700
Subject: [PATCH 2/3] upstream patch: Cast a few parameters when translating
macros to functions

original patch merged to 5.0.2 version
https://github.com/open-mpi/ompi/issues/12169

When we made the transition from macros to functions, we
were forced to move things that went on the left side of
an `=` sign to being addresses of params to the function
call. Compilers are getting increasinly picky about matching
parameter types, so cast a few that might cause trouble.

Signed-off-by: Ralph Castain <rhc@pmix.org>

Fixes following error:

/home/chris/rocm_sdk_builder/src_projects/openmpi/3rd-party/openpmix/include/pmix_deprecated.h:851:32: error: passing argument 2 of 'PMIx_Data_buffer_unload' from incompatible pointer type [-Wincompatible-pointer-types]
851 | PMIx_Data_buffer_unload(b, &(d), &(s))
| ^~~~
| |
| void **
/home/chris/rocm_sdk_builder/src_projects/openmpi/oshmem/mca/memheap/base/memheap_base_mkey.c:451:5: note: in expansion of macro 'PMIX_DATA_BUFFER_UNLOAD'
451 | PMIX_DATA_BUFFER_UNLOAD(msg, buffer, size);
| ^~~~~~~~~~~~~~~~~~~~~~~
/home/chris/rocm_sdk_builder/src_projects/openmpi/3rd-party/openpmix/include/pmix_deprecated.h:352:49: note: expected 'char **' but argument is of type 'void **'
352 | char **bytes, size_t *sz);
| ~~~~~~~^~~~~
/home/chris/rocm_sdk_builder/src_projects/openmpi/oshmem/mca/memheap/base/memheap_base_mkey.c: In function 'mca_memheap_modex_recv_all':
/home/chris/rocm_sdk_builder/src_projects/openmpi/3rd-party/openpmix/include/pmix_deprecated.h:851:32: error: passing argument 2 of 'PMIx_Data_buffer_unload' from incompatible pointer type [-Wincompatible-pointer-types]
851 | PMIx_Data_buffer_unload(b, &(d), &(s))
| ^~~~
| |
| void **
/home/chris/rocm_sdk_builder/src_projects/openmpi/oshmem/mca/memheap/base/memheap_base_mkey.c:586:5: note: in expansion of macro 'PMIX_DATA_BUFFER_UNLOAD'
586 | PMIX_DATA_BUFFER_UNLOAD(msg, send_buffer, size);
| ^~~~~~~~~~~~~~~~~~~~~~~
/home/chris/rocm_sdk_builder/src_projects/openmpi/3rd-party/openpmix/include/pmix_deprecated.h:352:49: note: expected 'char **' but argument is of type 'void **'
352 | char **bytes, size_t *sz);

Signed-off-by: Mika Laitio <lamikr@gmail.com>
---
3rd-party/openpmix/include/pmix_deprecated.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/3rd-party/openpmix/include/pmix_deprecated.h b/3rd-party/openpmix/include/pmix_deprecated.h
index 254dfb2..b5717ae 100644
--- a/3rd-party/openpmix/include/pmix_deprecated.h
+++ b/3rd-party/openpmix/include/pmix_deprecated.h
@@ -508,10 +508,10 @@ PMIX_EXPORT pmix_info_t* PMIx_Info_list_get_info(void *ptr, void *prev, void **n
(r) = PMIx_Argv_count(a)

#define PMIX_ARGV_APPEND(r, a, b) \
- (r) = PMIx_Argv_append_nosize(&(a), (b))
+ (r) = PMIx_Argv_append_nosize((char***)&(a), (b))

#define PMIX_ARGV_PREPEND(r, a, b) \
- (r) = PMIx_Argv_prepend_nosize(&(a), b)
+ (r) = PMIx_Argv_prepend_nosize((char***)&(a), b)

#define PMIX_ARGV_APPEND_UNIQUE(r, a, b) \
(r) = PMIx_Argv_append_unique_nosize(a, b)
@@ -848,7 +848,7 @@ PMIX_EXPORT pmix_info_t* PMIx_Info_list_get_info(void *ptr, void *prev, void **n
PMIx_Data_buffer_load(b, d, s)

#define PMIX_DATA_BUFFER_UNLOAD(b, d, s) \
- PMIx_Data_buffer_unload(b, &(d), &(s))
+ PMIx_Data_buffer_unload(b, (char**)&(d), (size_t*)&(s))

#define PMIX_PROC_CREATE(m, n) \
(m) = PMIx_Proc_create(n)
--
2.45.1

273 changes: 273 additions & 0 deletions patches/rocm-6.1.1/openmpi/0003-added-construct_event_strings.py-thats-missing-tar.g.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,273 @@
From 72ba0a6d531ab0cc325895bf57e5fba2edebd591 Mon Sep 17 00:00:00 2001
From: Mika Laitio <lamikr@gmail.com>
Date: Wed, 29 May 2024 14:15:28 -0700
Subject: [PATCH 3/3] added construct_event_strings.py thats missing tar.gz

patching of git submodules does not work well
yet with the rcom sdk builder, therefore openmpi
and all it's submodules has been put to single repo.

release openmpi-5.0.1.tar.bz2 used as a source however missed the
construct_event_strings.py.

Another patch just added for gcc14 updating the headers
seems to trigger the usage of this python file.
(Earlier it compiled ok because this python script was not tried to call)

Signed-off-by: Mika Laitio <lamikr@gmail.com>
---
.../contrib/construct_event_strings.py | 242 ++++++++++++++++++
1 file changed, 242 insertions(+)
create mode 100755 3rd-party/openpmix/contrib/construct_event_strings.py

diff --git a/3rd-party/openpmix/contrib/construct_event_strings.py b/3rd-party/openpmix/contrib/construct_event_strings.py
new file mode 100755
index 0000000..52f242f
--- /dev/null
+++ b/3rd-party/openpmix/contrib/construct_event_strings.py
@@ -0,0 +1,242 @@
+#
+# Copyright (c) 2020 Intel, Inc. All rights reserved.
+# Copyright (c) 2020-2022 Cisco Systems, Inc. All rights reserved
+# Copyright (c) 2021-2023 Nanook Consulting. All rights reserved.
+# Copyright (c) 2022 Triad National Security, LLC. All rights reserved.
+# $COPYRIGHT$
+#
+# Construct a dictionary for translating attributes to/from
+# their defined name and their string representation - used
+# by tools to interpret user input
+#
+
+from __future__ import print_function
+import os
+import os.path
+import sys
+from optparse import OptionParser, OptionGroup
+
+index = 0
+
+
+def harvest_constants(options, path, constants):
+ global index
+ # open the file
+ try:
+ inputfile = open(path, "r")
+ except Exception as e:
+ print("File {path} could not be opened: {e}"
+ .format(path=path, e=e))
+ return 1
+
+ # read the file - these files aren't too large
+ # so ingest the whole thing at one gulp
+ try:
+ lines = inputfile.readlines()
+ except Exception as e:
+ print("Error reading file {path}: {e}"
+ .format(path=path, e=e))
+ inputfile.close()
+ return 1
+
+ inputfile.close() # we read everything, so done with the file
+
+ firstline = True
+
+ # find the start of the event codes
+ n = 0
+ found = False
+ while n < len(lines):
+ if "PMIX ERROR CONSTANTS" in lines[n]:
+ found = True
+ n = n + 1
+ break;
+ n = n + 1
+ # error out if not found
+ if not found:
+ print("START OF EVENT CODES NOT FOUND")
+ return 1
+
+ # loop over the lines
+ while n < len(lines):
+ line = lines[n]
+ # remove white space at front and back
+ myline = line.strip()
+ # remove comment lines
+ if "/*" in myline or "*/" in myline or myline.startswith("*"):
+ n = n + 1
+ continue
+ # if we have found the end of the event codes, we are done
+ if "PMIX_EXTERNAL_ERR_BASE" in myline or "PMIX_ERR_SYS_BASE" in myline or "PMIX_INTERNAL_ERR_DONE" in myline:
+ return 0
+ # skip a well-known macro
+ if "PMIX_SYSTEM_EVENT" in myline:
+ n = n + 2
+ continue
+ # if the line starts with #define, then we want it
+ if not myline.startswith("#define"):
+ n = n + 1
+ continue
+
+ value = myline[8:]
+ tokens = value.split()
+ if not firstline:
+ constants.write(",\n\n")
+ firstline = False
+ constants.write(" {.index = " + str(index) + ", .name = \"" + tokens[0] + "\", .code = " + str(tokens[1]) + "}")
+ index = index + 1
+ n = n + 1
+
+ return 0
+
+
+def _write_header(options, base_path, num_elements):
+ contents = '''/*
+ * This file is autogenerated by construct_event_strings.py.
+ * Do not edit this file by hand.
+ */
+
+#include "src/include/pmix_config.h"
+#include "src/include/pmix_globals.h"
+#include "include/pmix_common.h"
+
+#ifndef PMIX_EVENT_STRINGS_H
+#define PMIX_EVENT_STRINGS_H
+
+BEGIN_C_DECLS
+
+PMIX_EXPORT extern const pmix_event_string_t pmix_event_strings[{ne}];
+
+#define PMIX_EVENT_INDEX_BOUNDARY {nem1}
+
+END_C_DECLS
+
+#endif\n
+'''.format(ne=num_elements, nem1=num_elements - 1)
+
+ if options.dryrun:
+ constants = sys.stdout
+ outpath = None
+ else:
+ outpath = os.path.join(base_path, "pmix_event_strings.h")
+ try:
+ constants = open(outpath, "w+")
+ except Exception as e:
+ print("{outpath} CANNOT BE OPENED - EVENT STRINGS COULD NOT BE CONSTRUCTED: {e}"
+ .format(outpath=outpath, e=e))
+ return 1
+ constants.write(contents)
+ constants.close()
+ return 0
+
+
+def main():
+ parser = OptionParser("usage: %prog [options]")
+ debugGroup = OptionGroup(parser, "Debug Options")
+ debugGroup.add_option("--dryrun",
+ action="store_true", dest="dryrun", default=False,
+ help="Show output to screen")
+ parser.add_option_group(debugGroup)
+
+ (options, args) = parser.parse_args()
+
+ # Find the top-level PMIx source tree dir.
+ # Start with the location of this script, which we know is in
+ # $top_srcdir/contrib.
+ top_src_dir = os.path.dirname(sys.argv[0])
+ top_src_dir = os.path.join(top_src_dir, "..")
+ top_src_dir = os.path.abspath(top_src_dir)
+
+ # Sanity check
+ checkfile = os.path.join(top_src_dir, "VERSION")
+ if not os.path.exists(checkfile):
+ print("ERROR: Could not find top source directory for Open PMIx")
+ return 1
+
+ source_include_dir = os.path.join(top_src_dir, "include")
+ util_include_dir = os.path.join(top_src_dir, "src", "util")
+
+ # This script is invoked from src/include/Makefile.am, and
+ # therefore the cwd will be $(builddir)/src/include. Verify this
+ # by checking for a file that we know should be in there.
+ build_src_include_dir = os.getcwd()
+ checkfile = os.path.join(build_src_include_dir, "pmix_config.h")
+ if not os.path.exists(checkfile):
+ print("ERROR: Could not find build directory for Open PMIx")
+ return 1
+
+ if options.dryrun:
+ constants = sys.stdout
+ outpath = None
+ else:
+ outpath = os.path.join(build_src_include_dir, "pmix_event_strings.c")
+ try:
+ constants = open(outpath, "w+")
+ except Exception as e:
+ print("{outpath} CANNOT BE OPENED - EVENT STRINGS COULD NOT BE CONSTRUCTED: {e}"
+ .format(outpath=outpath, e=e))
+ return 1
+
+ # write the source file
+ constants.write("""/*
+ * This file is autogenerated by construct_event_strings.py.
+ * Do not edit this file by hand.
+ */
+
+#include "src/include/pmix_event_strings.h"
+
+const pmix_event_string_t pmix_event_strings[] = {
+""")
+
+ # scan across the header files in the src directory
+ # looking for events
+
+ # pmix_common.h.in is in the src tree
+ rc = harvest_constants(options,
+ os.path.join(source_include_dir, "pmix_common.h.in"),
+ constants)
+ if 0 != rc:
+ constants.close()
+ if outpath:
+ os.remove(outpath)
+ print("HARVEST PMIX_COMMON FAILED - EVENT STRINGS COULD NOT BE CONSTRUCTED")
+ return 1
+ constants.write(",\n\n")
+
+ # pmix_deprecated.h is in the source tree
+ rc = harvest_constants(options,
+ os.path.join(source_include_dir, "pmix_deprecated.h"),
+ constants)
+ if 0 != rc:
+ constants.close()
+ if outpath:
+ os.remove(outpath)
+ print("HARVEST PMIX_DEPRECATED FAILED - EVENT STRINGS COULD NOT BE CONSTRUCTED")
+ return 1
+ constants.write(",\n\n")
+
+ # pmix_error.h is in the source tree
+ rc = harvest_constants(options,
+ os.path.join(util_include_dir, "pmix_error.h"),
+ constants)
+ if 0 != rc:
+ constants.close()
+ if outpath:
+ os.remove(outpath)
+ print("HARVEST PMIX_ERROR FAILED - EVENT STRINGS COULD NOT BE CONSTRUCTED")
+ return 1
+
+ # mark the end of the array
+ constants.write(""",\n
+ {.index = UINT32_MAX, .name = "", .code = -1}
+};
+""")
+ constants.write("\n")
+ constants.close()
+
+ # write the header
+ return _write_header(options, build_src_include_dir, index + 1)
+
+
+if __name__ == '__main__':
+ exit(main())
--
2.45.1

0 comments on commit b95d8b7

Please sign in to comment.