Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix rebuild for python3.12 update #65

Closed
wants to merge 4 commits into from

Conversation

raulcd
Copy link
Member

@raulcd raulcd commented Oct 16, 2023

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

I am just trying to fix the build from the auto-tick for Python 3.12

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@raulcd
Copy link
Member Author

raulcd commented Oct 16, 2023

When building the new Apache Arrow 14.0.0 release I found that our tests for conda-python 3.12 were failing due to gdb not being updated yet. As I've seen the error that appeared on CI before:

Traceback (most recent call last):
  File "/Users/runner/miniforge3/conda-bld/gdb_1695945122800/work/gdb/python/python-config.py", line 7, in <module>
    from distutils import sysconfig
ModuleNotFoundError: No module named 'distutils'

Which is related Python 3.12 and this setuptools issue: pypa/setuptools#3661
I have tried to fix it on this PR.
Thanks!

@raulcd
Copy link
Member Author

raulcd commented Oct 16, 2023

ok, the error is solved but there are other issues related to the Python 3.12 builds. I currently don't have the time to investigate so feel free to close this PR and use or disregard my suggestion.

@jakirkham
Copy link
Member

No need to close. This can be a useful point for someone else to get started 😉

Thanks for getting this work going! 🙏

recipe/meta.yaml Outdated Show resolved Hide resolved
@jorisvandenbossche
Copy link
Member

The python test on Python 3.12 gives the follow gdb output:

Program received signal SIGSEGV, Segmentation fault.
0x00007f807219c657 in kill () from /lib64/libc.so.6
Traceback (most recent call first):
  <optimized out>
  File "/tmp/tmpctni8yos/info/recipe/testing/process_to_debug.py", line 3, in <module>
    os.kill(os.getpid(), signal.SIGSEGV)

while comparing this with the Python 3.11 logs, I see:

Program received signal SIGSEGV, Segmentation fault.
0x00007fc0ff418657 in kill () from /lib64/libc.so.6
Traceback (most recent call first):
  <built-in method kill of module object at remote 0x7fc0ff191760>
  File "/tmp/tmpqvo6o9q7/info/recipe/testing/process_to_debug.py", line 3, in <module>
    os.kill(os.getpid(), signal.SIGSEGV)
  <built-in method kill of module object at remote 0x7fc0ff191760>
  File "/tmp/tmpqvo6o9q7/info/recipe/testing/process_to_debug.py", line 3, in <module>
  File "/tmp/tmpqvo6o9q7/info/recipe/testing/process_to_debug.py", line 3, in <module>
    os.kill(os.getpid(), signal.SIGSEGV)
Program received signal SIGSEGV, Segmentation fault.

I don't know if that is indicating a problem (if that output shouldn't have changed), but in case that change is fine (eg because of a Python change), the script asserting some things being present in the output probably needs to be updated. For example:

gdb -batch -ex "run" -ex "py-bt" --args python "$RECIPE_DIR/testing/process_to_debug.py" | tee gdb_output
if [[ "$CONDA_PY" != "27" ]]; then
grep "built-in method kill" gdb_output
fi

built-in method kill is no longer in the output.

Maybe the Python3.12 build on conda-forge has less debug symbols?

@Gunni
Copy link

Gunni commented Jan 17, 2024

FYI

Python 3.12.0 alpha 7

Release date: 2023-04-04

gh-102809: Misc/gdbinit was removed.

@mikev-junimo
Copy link

What is the reason new gdb builds aren't produced in conda-forge? Is this the python 3.12 compatibility?

@phil-blain
Copy link
Contributor

The python test on Python 3.12 gives the follow gdb output:

Program received signal SIGSEGV, Segmentation fault.
0x00007f807219c657 in kill () from /lib64/libc.so.6
Traceback (most recent call first):
  <optimized out>
  File "/tmp/tmpctni8yos/info/recipe/testing/process_to_debug.py", line 3, in <module>
    os.kill(os.getpid(), signal.SIGSEGV)

while comparing this with the Python 3.11 logs, I see:

Program received signal SIGSEGV, Segmentation fault.
0x00007fc0ff418657 in kill () from /lib64/libc.so.6
Traceback (most recent call first):
  <built-in method kill of module object at remote 0x7fc0ff191760>
  File "/tmp/tmpqvo6o9q7/info/recipe/testing/process_to_debug.py", line 3, in <module>
    os.kill(os.getpid(), signal.SIGSEGV)
  <built-in method kill of module object at remote 0x7fc0ff191760>
  File "/tmp/tmpqvo6o9q7/info/recipe/testing/process_to_debug.py", line 3, in <module>
  File "/tmp/tmpqvo6o9q7/info/recipe/testing/process_to_debug.py", line 3, in <module>
    os.kill(os.getpid(), signal.SIGSEGV)
Program received signal SIGSEGV, Segmentation fault.

I don't know if that is indicating a problem (if that output shouldn't have changed), but in case that change is fine (eg because of a Python change), the script asserting some things being present in the output probably needs to be updated. For example:

gdb -batch -ex "run" -ex "py-bt" --args python "$RECIPE_DIR/testing/process_to_debug.py" | tee gdb_output
if [[ "$CONDA_PY" != "27" ]]; then
grep "built-in method kill" gdb_output
fi

built-in method kill is no longer in the output.

Maybe the Python3.12 build on conda-forge has less debug symbols?

Indeed, it seems the 3.12 build has less debug symbols. Simply moving the grep call for "builtin method kill" inside the else clause should be sufficient, along with listing 3.12 as undebugable:

diff --git a/recipe/run_test.sh b/recipe/run_test.sh
index da615bb..6c40e98 100644
--- a/recipe/run_test.sh
+++ b/recipe/run_test.sh
@@ -48,9 +48,6 @@ if [[ $(uname -m) == "ppc64le" ]]; then
 fi
 
 gdb -batch -ex "run" -ex "py-bt" --args python "$RECIPE_DIR/testing/process_to_debug.py" | tee gdb_output
-if [[ "$CONDA_PY" != "27" ]]; then
-    grep "built-in method kill" gdb_output
-fi
 
 # Unfortunately some python packages do not have enough debug info for py-bt
 #
@@ -64,7 +61,7 @@ fi
 # debugged out-of-the-box with this gdb package. When things change, there is not much to be
 # done besides adding or removing versions from this list.
 # Example: insufficient_debug_info_versions=("27" "37")
-insufficient_debug_info_versions=()
+insufficient_debug_info_versions=("312")
 
 if [[ " ${insufficient_debug_info_versions[@]} " =~ " ${CONDA_PY} " ]]; then
     if grep "line 3" gdb_output; then
@@ -75,6 +72,9 @@ if [[ " ${insufficient_debug_info_versions[@]} " =~ " ${CONDA_PY} " ]]; then
 else
     # We are lucky! This Python version has enough debug info for us to easily identify
     # the exact Python code where the crash happened.
+    if [[ "$CONDA_PY" != "27" ]]; then
+        grep "built-in method kill" gdb_output
+    fi
     grep "line 3" gdb_output
     grep "process_to_debug.py" gdb_output
     grep 'os.kill(os.getpid(), signal.SIGSEGV)' gdb_output

@phil-blain
Copy link
Contributor

What is the reason new gdb builds aren't produced in conda-forge? Is this the python 3.12 compatibility?

No, simply that I do not have a lot of time to maintain the feedstock, and the other maintainers are not really implicated anymore. Plus, I do not myself use the conda-forge build of GDB anymore as I find it's simpler to build it from scratch.

@mikev-junimo
Copy link

What is the reason new gdb builds aren't produced in conda-forge? Is this the python 3.12 compatibility?

No, simply that I do not have a lot of time to maintain the feedstock, and the other maintainers are not really implicated anymore. Plus, I do not myself use the conda-forge build of GDB anymore as I find it's simpler to build it from scratch.

Is there anything I can help with to push this forward?
We basically rely on conda-forge packages instead of distro packages so getting this working again would be great.
The problem is that I have zero knowledge of conda-forge packaging processes / best practices / etc...

@mikev-junimo
Copy link

Ah, looks like some work is being done in #69

@phil-blain
Copy link
Contributor

Thank you @raulcd, Python 3.12 was added in #70, I'll close this one.

@phil-blain phil-blain closed this Feb 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants