-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support DEPENDS_ARGS/FLAVORS - Building multiple packages from one origin #483
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Such as net/tiny-network-utilities which Poudriere previously failed to build on.
Mapping an existing package with an origin back to what origin and PKGNAME might be in the queue now is not straight forward because the origin may be in it with any number of DEPENDS_ARGS/FLAVOR to produce multiple packages. What we can do is look for a matching PKGBASE at the known origin. From here we take the closet matching PKGBASE as the new PKGNAME.
This is ugly and needs some better handling, but we need to display at least PKGNAME to make it more clear what is being displayed since 1 origin may now produce multiple packages. It would be enough to just show PKGNAME but it is far less convenient.
…pecs. The default originspec is now looked up first. Any dependency wanting to lookup a specific originspec (DEPENDS_ARGS or FLAVOR) is added to the flavorqueue and processed later.
This fixes the -a case and avoids writing extra files.
This will push non-default originspecs into the depqueue.
…f gather_port_vars() For the non -a case still assume that all flavors of an origin are to be considered as listed.
Conflicts: src/share/poudriere/common.sh
…orts. - Trim them from the build queue - Replace them as dependencies to the main port which supports DEPENDS_ARGS and FLAVORS
This also allows testing on virtual (non-existent) py3 ports like textproc/py3-hexdump, but it otherwise is required to be able to properly build and map ports like net/py3-netifaces.
Other things were refactored to use pkgname and cached lists for that.
…_ARE_FATAL. Consider bad dependency lines on the wrong PKGNAME as fatal. For example: BUILD_DEPENDS= p5-List-MoreUtils>=0:lang/p5-List-MoreUtils If this port's PKGNAME were really "List-MoreUtils" then it would not be recorded into the resulting package. The next build with CHECK_CHANGED_DEPS enabled would consider it a "new dependency" since it is in the the port but not in the package. This is usually a warning but can be made fatal instead by enabling this option.
…LAVORS. This also reworks some of the previously added assertions that FLAVORS cannot be combined with DEPENDS_ARGS since we don't ever lookup both now.
Conflicts: src/share/poudriere/common.sh
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #259 and #313, and obsoletes #314.
Remaining items:
The next thing will be to add FLAVORS support for https://reviews.freebsd.org/D10327. This should mostly be trivial.Support ORIGIN@FLAVOR syntax forbulk -f list
andbulk port
, etc vialisted_ports
Ensure that ORIGIN ORIGIN@FLAVOR1 ORIGIN@FLAVOR2 works together.
Support RUN_DEPENDS+= foo>0:devel/foo:FLAVOR=flaveMakeis_bad_flavor_slave_port
requeue the main port with the properDEPENDS_ARGS=PYTHON_VERSION
for the originally request port. This will relieve the pressure to commit FLAVORS support immediately to prevent breaking all of the python ports listed.The chosen FLAVOR or DEPENDS_ARGS is not always clear in the web interface but should be computable from the log. Not sure I will modify this before merge.(Will handle in JSON/HTML tracks by origin but should be pkgname #415)Allcache_get_origin
need auditted to ensure they shouldn't be using originspec, like bulk showing ports to build.The reinjection of items into the queue for DEPENDS_ARGS and FLAVORS needs to be unified for simplicity.moved to FLAVORS and DEPENDS_ARGS support is insane #492Similar to (7), FLAVORS and DEPENDS_ARGS should not be supported set together.Supportorigin@-
to only build the first default flavor.Need a test suite desperately. There's so many test cases here that keep failing when I change things. Need automated bulk tests #488Fix theCHECK_CHANGED_DEPS
support for ORIGIN@FLAVOR dependency lines. It currently will bomb since the packages only record ORIGIN. Need to convert to PKGBASE or something. This code belongs in ports badly.CHECKED_CHANGED_DEPS
support for originspecs #495The python slave port issue is quite complex.If a port like net/tiny-network-utilities requires net/py-netifaces as python3 it currently does not work due to the DEPENDS_ARGS bug. The workaround today in ports is to change the dependency to a new slave port such as net/py3-netifaces. In this specific example net/tiny-network-utilities does not depend on a py3 slaveport. Once Poudriere is merged with DEPENDS_ARGS support it, with
bulk -a
will then queue the virtual net/py-netifaces for python2 (default) and then also python3 from the net/tiny-network-utilities DEPENDS_ARGS dependency. It will also queue a build for net/py3-netifaces. The slave port and the dependency-generated package will both be 'py3-netifaces' for PKGNAME and thus hit the duplicated PKGNAME assertion and fail to build.To work around this I force mapped origins that had a slave port added but would build just fine now from the main port from a DEPENDS_ARGS-generated package. The problem that comes from all of this is that if a py3 slave port was added not for a dependency solution but purely to provide a py3 package for a user. In that case without FLAVORS a py3- package will never generate from the default port. So merging this into the package builds will drop a lot of py3- packages. That is fixed by merging FLAVORS support to the python ports which then will ensure all main python ports provide a py2 and py3 package from the origin whether they are a dependency or not.
Performance of
bulk -a
startup is hurt by 7% but it's relatively fast still.