Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-package modules. #258

Merged
merged 8 commits into from
Oct 7, 2022
Merged

Conversation

jaraco
Copy link
Member

@jaraco jaraco commented Jul 22, 2022

@jaraco jaraco requested a review from warsaw July 22, 2022 18:21
@jaraco
Copy link
Member Author

jaraco commented Jul 22, 2022

@warsaw I'd appreciate your review of this change, especially since you were involved so much in the discussion on #60. The behavior change is entirely in the second commit ecffc07. Note how it simplifies the code, aligns with pkg_resources behavior, and enables use-cases that were otherwise unreachable (resources for non-package modules). Is there any reason we shouldn't allow users to supply a module as an indicator for where to locate resources?

@jaraco
Copy link
Member Author

jaraco commented Jul 23, 2022

See jaraco/openpack@82a5d0b for an illustration of how this change simplifies and harmonizes resource loading.

_path.build(spec, self.site_dir)
import mod

actual = resources.files(mod).joinpath('res.txt').read_text()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is surprising to me. I would expect, a pathlib.Path('...', 'mod.py') to be returned, requiring resources.files(mod).parent.joinpath('res.txt'). res.txt is not a resource of mod, but rather of the parent module, which might not even exist if mod is top level.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sharing your surprise. Surely this would need some improved documentation to illustrate how files() now accepts an Anchor (instead of Package), where the anchor indicates:

  • the container (importer) from where resources can be discovered,
  • if a module is indicated, that's equivalent to indicating the container of that module,
  • if a top-level module is indicated, that will anchor on resources reachable at the top level by that same importer.

This is the behavior indicated by pkg_resources:

Note that if a module name is used, then the resource name is relative to the package immediately containing the named module.

However, I've also observed that the pkg_resources implementation also allows for loading resources against unpackaged modules. And while that behavior may have been unintentional, it works because it's a natural consequence of relying on the importer/loader to resolve the resources from the same location as the indicated package/module. That is, because the importlib mechanism supports loading modules from outside packages, so too should resources be loadable from that location.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

res.txt is not a resource of mod, but rather of the parent module, which might not even exist if mod is top level.

A parent module may not exist, but a loader for that module does exist and can load resources adjacent to that module:

 importlib_resources feature/203-non-package-modules $ touch setup.py
 importlib_resources feature/203-non-package-modules $ .tox/python/bin/python -iq
>>> import importlib_resources as res
>>> files = res.files('setup')
>>> files
PosixPath('/Users/jaraco/code/public/importlib_resources')
>>> files.joinpath('pyproject.toml').read_text()[:10]
'[build-sys'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 1f3c226, I've updated the documentation to honor an Anchor type which need not be a package.

@jaraco jaraco force-pushed the feature/203-non-package-modules branch from 1923197 to 1f3c226 Compare October 5, 2022 14:26
@jaraco jaraco force-pushed the feature/203-non-package-modules branch from 1f3c226 to 8c3edeb Compare October 5, 2022 14:30
@jaraco jaraco force-pushed the feature/203-non-package-modules branch from 2ff7b03 to 4e63613 Compare October 7, 2022 17:29
@jaraco jaraco merged commit 19ca481 into main Oct 7, 2022
@jaraco jaraco deleted the feature/203-non-package-modules branch October 7, 2022 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Unable to load resources for modules
2 participants