Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: set __file__ on submodules #5584

Merged
merged 6 commits into from
Mar 28, 2025
Merged

Conversation

henryiii
Copy link
Collaborator

@henryiii henryiii commented Mar 27, 2025

Description

The docs state 'The caller is responsible for setting a __file__ attribute', but we
were not doing that, which interferes with pickling objects in submodules using cloudpickle. Users previously had to set the __file__ attributes manually,
such as in https://github.com/scikit-hep/boost-histogram/blob/1fbbe1632e1665863b9c84b10edf6aa659a14bf1/src/boost_histogram/histogram.py#L83-L90.

Suggested changelog entry:

* Set ``__file__`` on submodules.

The docs state ['The caller is responsible for setting a `__file__` attribute'](https://docs.python.org/3/c-api/module.html), but we
were not doing that, which interferes with pickling objects in submodules using cloudpickle. Users previously had to set the `__file__` attributes manually,
such as in https://github.com/scikit-hep/boost-histogram/blob/1fbbe1632e1665863b9c84b10edf6aa659a14bf1/src/boost_histogram/histogram.py#L83-L90.

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@rwgk
Copy link
Collaborator

rwgk commented Mar 27, 2025

Traceback (most recent call last):
Function not implemented in GraalPy: PyModule_GetFilenameObject
  File "<string>", line 1, in <module>

@henryiii
Copy link
Collaborator Author

henryiii commented Mar 27, 2025

@msimacek, PyModule_GetFilenameObject is not implemented in GraalPy, it seems. There is the older and deprecated PyModule_GetFilename (both are part of the Stable ABI), or we could try to get the __file__ attr, or we could just skip this on GraalPy.

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@@ -1336,6 +1336,11 @@ class module_ : public object {
if (doc && options::show_user_defined_docstrings()) {
result.attr("__doc__") = pybind11::str(doc);
}

// GraalPy doesn't support PyModule_GetFilenameObject,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine if you want to keep it, but the official PyModule_GetFilenameObject() seems most future proof, compared to referencing "__file__" directly.

We have 7 defined(GRAALVM_PYTHON) already under include/pybind11/, I'd lean towards adding another one here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, done.

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@msimacek
Copy link
Contributor

@msimacek, PyModule_GetFilenameObject is not implemented in GraalPy, it seems. There is the older and deprecated PyModule_GetFilename (both are part of the Stable ABI), or we could try to get the __file__ attr, or we could just skip this on GraalPy.

Thank you for the notification. I'll add the function for the next version of GraalPy

@rwgk
Copy link
Collaborator

rwgk commented Mar 28, 2025

@henryiii From my end, this is good to merge anytime.

@msimacek Is there a version number we could check for? — That's the best way to know later if we can purge the workaround or not yet.

@msimacek
Copy link
Contributor

Yes, if I fix it for GraalPy 25.0 you could do #ifdef GRAALVM_PYTHON && GRAALPY_VERSION_NUM < 0x190000

@@ -1336,6 +1336,19 @@ class module_ : public object {
if (doc && options::show_user_defined_docstrings()) {
result.attr("__doc__") = pybind11::str(doc);
}

#ifdef GRAALVM_PYTHON
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#ifdef GRAALVM_PYTHON
#if defined(GRAALVM_PYTHON) && GRAALPY_VERSION_NUM < 0x190000

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion applied with commit 4682e14

rwgk added 3 commits March 28, 2025 09:01
```
/Users/runner/work/pybind11/pybind11/include/pybind11/pybind11.h:1340:32: error: 'GRAALPY_VERSION_NUM' is not defined, evaluates to 0 [-Werror,-Wundef]
                               ^
```

Related ChatGPT conversation: https://chatgpt.com/share/67e6cb99-84b0-8008-99d6-aadc70242cf3
@rwgk rwgk merged commit 8f00d1e into pybind:master Mar 28, 2025
74 checks passed
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label Mar 28, 2025
@henryiii henryiii removed the needs changelog Possibly needs a changelog entry label Apr 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants