Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Python detection optional and more portable #8731

Merged
merged 1 commit into from Jun 5, 2019
Merged

Make Python detection optional and more portable #8731

merged 1 commit into from Jun 5, 2019

Conversation

ghost
Copy link

@ghost ghost commented May 9, 2019

Summary

When detecting the Python version, don't assume the path to binaries.
Make sure --without-python works.

Motivation and Context

These changes make it easier to build the zfsonlinux code on platforms like FreeBSD, where true is /usr/bin/true and python3 is /usr/local/bin/python3 or might not exist at all (i.e. in the base system).

Description

Previously, --without-python would cause ./configure to fail. Now it is able to proceed, and the Python scripts will not be built.

Use portable regular expression matching instead of nonstandard substring expansion to detect the Python version. This test is duplicated in several places, so define a function for it.

Don't hard-code paths to binaries such as trueand python3.

The cmd/ subdirs have been split up to only build the Python subdirs when configured to use Python. Coincidentally, Linux-specific subdirs are also split out from the same line, reducing the diff from ZoF.

How Has This Been Tested?

These changes are based on work in progress in porting zol to FreeBSD. I have built ZFS on FreeBSD with these changes.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (a change to man pages or other documentation)

Checklist:

@rlaager
Copy link
Member

rlaager commented May 27, 2019

This should fix #8809.

Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you tackling this portability issue. The approach taken is nice, I did run in to a few minor issues which need to be sorted out (see comments). When updating this PR if you can rebase it on master with the proposed tweaks it should pass the CI cleanly.

config/always-python.m4 Outdated Show resolved Hide resolved
config/always-python.m4 Outdated Show resolved Hide resolved
config/always-pyzfs.m4 Show resolved Hide resolved
Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for refreshing this. I happened to run in to one other minor issue when testing this locally, see the USING_PYTHON comment.

config/always-pyzfs.m4 Outdated Show resolved Hide resolved
@behlendorf behlendorf requested review from rlaager and ofaaland May 30, 2019 13:54
cmd/Makefile.am Show resolved Hide resolved
config/always-python.m4 Outdated Show resolved Hide resolved
Copy link
Member

@rlaager rlaager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM. I'll leave it to @freqlabs to determine if the test + which thing should be replaced by just which.

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Jun 3, 2019
@ghost
Copy link
Author

ghost commented Jun 3, 2019

@behlendorf If it's alright, I'd like to push a revision inspired by @eli-schwartz's suggestion.

@behlendorf
Copy link
Contributor

@freqlabs absolutely, please go ahead update the PR.

@behlendorf behlendorf added Status: Code Review Needed Ready for review and testing and removed Status: Accepted Ready to integrate (reviewed, tested) labels Jun 4, 2019
Previously, --without-python would cause ./configure to fail. Now it is
able to proceed, and the Python scripts will not be built.

Use portable parameter expansion matching instead of nonstandard
substring matching to detect the Python version.  This test is
duplicated in several places, so define a function for it.

Don't assume the full path to binaries, since different platforms do
install things in different places.  Use AC_CHECK_PROGS instead.

When building without Python, also build without pyzfs.

Sponsored by: iXsystems, Inc.
Signed-off-by: Ryan Moeller <ryan@freqlabs.com>
@codecov
Copy link

codecov bot commented Jun 4, 2019

Codecov Report

Merging #8731 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8731      +/-   ##
==========================================
+ Coverage   78.67%   78.67%   +<.01%     
==========================================
  Files         382      382              
  Lines      117771   117780       +9     
==========================================
+ Hits        92651    92659       +8     
- Misses      25120    25121       +1
Flag Coverage Δ
#kernel 79.28% <ø> (-0.03%) ⬇️
#user 67.1% <ø> (-0.32%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df24bcf...1a46b4b. Read the comment docs.

Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This turned out nicely. Unless, anyone has additional changes to suggest this is ready to merge.

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Jun 4, 2019
@behlendorf behlendorf merged commit 1a132f0 into openzfs:master Jun 5, 2019
AC_MSG_ERROR([Cannot find $PYTHON in your system path])
AS_IF([test $PYTHON != :], [
AS_IF([$PYTHON --version >/dev/null 2>&1],
[AM_PATH_PYTHON([2.6], [], [:])],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is actually redundant, since AM_PATH_PYTHON will already try to run $PYTHON -c some-inline-python-script-which-consults-sys.hexversion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, a bit late, hehe.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, well to remove the redundancy let's open a new PR.

behlendorf pushed a commit that referenced this pull request Jun 7, 2019
Previously, --without-python would cause ./configure to fail. Now it is
able to proceed, and the Python scripts will not be built.

Use portable parameter expansion matching instead of nonstandard
substring matching to detect the Python version.  This test is
duplicated in several places, so define a function for it.

Don't assume the full path to binaries, since different platforms do
install things in different places.  Use AC_CHECK_PROGS instead.

When building without Python, also build without pyzfs.

Sponsored by: iXsystems, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Eli Schwartz <eschwartz93@gmail.com>
Signed-off-by: Ryan Moeller <ryan@freqlabs.com>
Closes #8809 
Closes #8731
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants