Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow 'nosource: True' in PythonPackage extensions #3305

Merged

Conversation

Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Apr 12, 2024

(created using eb --new-pr)

To be used with easybuilders/easybuild-framework#4506

Motivation is a use case like: https://github.com/ComputeCanada/easybuild-easyconfigs/blob/5573c66a91fb2990723da630e7eb690957a59f46/easybuild/easyconfigs/d/DuckDB/DuckDB-0.10.1-GCCcore-12.3.0.eb

easyblock = 'CMakeNinja'

...

exts_defaultclass = 'PythonPackage'
exts_list = [
    ("%(namelower)s", version, {
        'nosource': True, # bundled already
        'start_dir': 'tools/pythonpkg/',
        'preconfigopts': 'export DUCKDB_BINARY_DIR=%(installdir)s &&',
        'use_pip': True,
    }),
]

I.e. we want to build using some build system which has a python package included or as a build artifact. Hence we don't need nor want to use a source.

With these 2 PRs it gets to the correct pip install . in the right directory. I couldn't test further as it fails to get the version using setuptools_scm which tries to use git on a source-only tarball.

@smoors
Copy link
Contributor

smoors commented Apr 18, 2024

I couldn't test further as it fails to get the version using setuptools_scm which tries to use git on a source-only tarball.

this can be worked around by setting SETUPTOOLS_SCM_PRETEND_VERSION envvar:

'preinstallopts': 'export SETUPTOOLS_SCM_PRETEND_VERSION=%(version)s && '

@Flamefire
Copy link
Contributor Author

this can be worked around by setting SETUPTOOLS_SCM_PRETEND_VERSION envvar:

'preinstallopts': 'export SETUPTOOLS_SCM_PRETEND_VERSION=%(version)s && '

Works, thanks!
This is the EC I used: DuckDB-0.10.1-GCCcore-12.3.0.eb

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS DuckDB-0.10.1-GCCcore-12.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
n1493 - Linux RHEL 8.7 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (icelake), Python 3.8.13
See https://gist.github.com/Flamefire/5e2eef06a838a56bf94b74c61b65aaef for a full test report.

@boegel
Copy link
Member

boegel commented Apr 18, 2024

I couldn't test further as it fails to get the version using setuptools_scm which tries to use git on a source-only tarball.

this can be worked around by setting SETUPTOOLS_SCM_PRETEND_VERSION envvar:

'preinstallopts': 'export SETUPTOOLS_SCM_PRETEND_VERSION=%(version)s && '

How insane would it be to let PythonPackage automatically set this environment variable when installing an extension?
Would that cause trouble?

@Flamefire
Copy link
Contributor Author

How insane would it be to let PythonPackage automatically set this environment variable when installing an extension? Would that cause trouble?

Hm, I can't think of any issues. We always want the version of the software as the version. And if this avoids even trying to use git (where either the binary or the .git folder may not exist) this would be two birds with one stone: Faster and less issues

@smoors
Copy link
Contributor

smoors commented Apr 18, 2024

Hm, I can't think of any issues. We always want the version of the software as the version. And if this avoids even trying to use git (where either the binary or the .git folder may not exist) this would be two birds with one stone: Faster and less issues

i agree. there will be slight differences in exceptional cases with a git checkout of a commit. for example, for GBprocesS/4.0.0.post1-foss-2022a i get

>>> import gbprocess
>>> gbprocess.__version__
'4.0.0.post2.dev0+g26119ab.d20240418'

in gitlab this commit is tagged as 4.0.0.post1, so i'd say our version is more useful in this case

@smoors smoors added this to the release after 4.9.1 milestone Apr 18, 2024
Copy link
Contributor

@smoors smoors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@smoors
Copy link
Contributor

smoors commented Apr 18, 2024

Going in, thanks @Flamefire!

@smoors smoors merged commit 9473538 into easybuilders:develop Apr 18, 2024
47 checks passed
@Flamefire Flamefire deleted the 20240412170740_new_pr_pythonpackage branch April 18, 2024 15:59
@boegel boegel changed the title Allow 'nosource: True' in PythonPackage extensions' Allow 'nosource: True' in PythonPackage extensions Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants