build: reduce autoconf input files from 32 to 2 #5219
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kmk3
force-pushed
the
build-reduce-config-files
branch
from
a2ecd4d
to
02510ea
Compare
|
To clarify, here is a summary of how the output files are generated from the
Firstly, the configure script is still generated in the same way as always:
As of firejail 0.9.70 (that is, prior to #5140), this is what happens:
->
That is, every makefile is generated, regardless of whether it depends on any With #5140 and this PR:
->
Which means that the output variables (such as |
reinerh
approved these changes
Jun 25, 2022
Configure summary: autoconf essentially only parses configure.ac and generates the configure script (that is, the "./configure" shell script). The latter is what actually checks what is available on the system and internally sets the value of the output variables. It then, for every filename foo in AC_CONFIG_FILES (and for every output variable name BAR in AC_SUBST), reads foo.in, replaces every occurrence of `@BAR@` with the value of the shell variable `$BAR` and generates the file foo from the result. After this, configure is finished and `make` could be executed to start the build. Now that (as of netblue30#5140) all output variables are only defined on config.mk.in and on config.sh.in, there is no need to generate any makefile nor any other mkfile or shell script at configure time. So rename every "Makefile.in" to "Makefile", mkdeb.sh.in to mkdeb.sh, src/common.mk.in to src/common.mk and leave just config.mk and config.sh as the files to be generated at configure time. This allows editing and committing all makefiles directly, without potentially having to run ./configure in between. Commands used to rename the makefiles: $ git ls-files -z -- '*Makefile.in' | xargs -0 -I '{}' sh -c \ "git mv '{}' \"\$(dirname '{}')/Makefile\"" Additionally, from my (rudimentary) testing, this commit reduces the time it takes to run ./configure by about 20~25% compared to commit 72ece92 ("Transmission fixes: drop private-lib (netblue30#5213)", 2022-06-22). Environment: dash 0.5.11.5-1, gcc 12.1.0-2, Artix Linux, ext4 on an HDD. Commands used for benchmarking each commit: $ : >time_configure && ./configure && make distclean && for i in $(seq 1 10); do { time -p ./configure; } 2>>time_configure; done $ grep real time_configure | awk '{ total += $2 } END { print total/NR }'
This allows running `make clean` and `make distclean` (and possibly
others) without having to run ./configure beforehand.
Note that some packaging-related targets still depend on the existence
of generated files. For example:
* dist: config.mk
* deb: config.sh
Commands used to search and replace:
$ git grep -Elz 'include *([^ ]*/)?config.mk' | xargs -0 -I '{}' \
sh -c "printf '%s\n' \
\"\$(sed -E 's|^include *(([^ ]*/)?config.mk)|-include \1|' '{}')\" >'{}'"
Relates to netblue30#5140.
With the previous commit ("makefiles: stop failing when config.mk does
not exist", 2022-06-23), make will not immediately fail when trying to
build a target without having the proper compile-time flags (which are
defined on common.mk).
For example, when running the command below:
make distclean && make
It will throw an error only after (mis-)compiling multiple objects.
So add a dependency on config.mk on every target that uses output
variables (such as @name@ / $(NAME)) on its recipe. And add a
dependency on config.sh on targets that call shell scripts that use
output variables (such as @name@ / $NAME). Also, add a recipe for
config.mk / config.sh telling to run ./configure, to make it a bit more
obvious just in case.
With this commit, make will abort earlier, by detecting that the
config.mk / config.sh dependency does not exist. This happens before
trying to execute the recipe.
This also makes the dependencies more accurate, since if config.mk
(which defines some CFLAGS) is changed, the CFLAGS may also have
changed, so a target that uses CFLAGS should probably be considered out
of date in this case anyway.
Relates to netblue30#5140.
kmk3
force-pushed
the
build-reduce-config-files
branch
from
02510ea
to
e21637c
Compare
|
Force-pushed to add more information to the first commit message, as I noticed |
|
all merged, thanks! |
kmk3
added a commit
to kmk3/firejail
that referenced
this pull request
Jul 12, 2022
The following leverages the fact that when using a normal merge (as
opposed to "rebase and merge" or "squash and merge") on GitHub, the pull
request number is put in the commit message title and the title of the
PR is added to the commit message body.
Commands used to find and print the items for the RELNOTES:
$ git log --grep='^build:' --merges --reverse --pretty='%s %b' 0.9.70.. |
sed -E -n 's/Merge pull request (#[0-9]+) from [^ ]+ (.*)/ * \2 (\1)/p'
* build: deduplicate configure-time vars into new config files (netblue30#5140)
* build: fix file mode of shell scripts (644 -> 755) (netblue30#5206)
* build: reduce autoconf input files from 32 to 2 (netblue30#5219)
Commands used to generate the message below:
$ git log --grep='^build:' --merges --reverse --pretty='%s %b' 0.9.70.. |
sed -E -n 's/Merge pull request (#[0-9]+).*/\1/p' | sort | tr '\n' ' ' |
sed -E 's/^(.*) /Relates to \1./'
Relates to netblue30#5140 netblue30#5206 netblue30#5219.
Relates to netblue30#5140 netblue30#5206 netblue30#5219.
kmk3
added a commit
that referenced
this pull request
Jul 12, 2022
The following leverages the fact that when using a normal merge (as
opposed to "rebase and merge" or "squash and merge") on GitHub, the pull
request number is put in the commit message title and the title of the
PR is added to the commit message body.
Commands used to find and print the items for the RELNOTES:
$ git log --grep='^build:' --merges --reverse --pretty='%s %b' 0.9.70.. |
sed -E -n 's/Merge pull request (#[0-9]+) from [^ ]+ (.*)/ * \2 (\1)/p'
* build: deduplicate configure-time vars into new config files (#5140)
* build: fix file mode of shell scripts (644 -> 755) (#5206)
* build: reduce autoconf input files from 32 to 2 (#5219)
Commands used to generate the message below:
$ git log --grep='^build:' --merges --reverse --pretty='%s %b' 0.9.70.. |
sed -E -n 's/Merge pull request (#[0-9]+).*/\1/p' | sort | tr '\n' ' ' |
sed -E 's/^(.*) /Relates to \1./'
Relates to #5140 #5206 #5219.
Relates to #5140 #5206 #5219.
kmk3
added a commit
to kmk3/firejail
that referenced
this pull request
Jan 28, 2023
And also add an "error: " prefix, to make the output clearer.
Before:
$ rm -f config.mk; make config.mk
printf 'run ./configure to generate %s\n' "config.mk" >&2
run ./configure to generate config.mk
false
make: *** No rule to make target 'config.mk'. Stop.
After:
$ rm -f config.mk; make config.mk
error: run ./configure to generate config.mk
make: *** No rule to make target 'config.mk'. Stop.
This amends commit e21637c ("makefiles: add generated files as
dependencies", 2022-06-23) / PR netblue30#5219.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Configure summary: autoconf essentially only parses configure.ac and
generates the configure script (that is, the "./configure" shell
script). The latter is what actually checks what is available on the
system and internally sets the value of the output variables. It then,
for every filename foo in AC_CONFIG_FILES (and for every output variable
name BAR in AC_SUBST), reads foo.in, replaces every occurrence of
@BAR@with the value of the shell variable$BARand generates thefile foo from the result. After this, configure is finished and
makecould be executed to start the build.
Now that (as of #5140) all output variables are only defined on
config.mk.in and on config.sh.in, there is no need to generate any
makefile nor any other mkfile or shell script at configure time. So
rename every "Makefile.in" to "Makefile", mkdeb.sh.in to mkdeb.sh,
src/common.mk.in to src/common.mk and leave just config.mk and config.sh
as the files to be generated at configure time.
This allows editing and committing all makefiles directly, without
potentially having to run ./configure in between.
Commands used to rename the makefiles:
Additionally, from my (rudimentary) testing, this commit reduces the
time it takes to run ./configure by about 20~25% compared to commit
72ece92 ("Transmission fixes: drop private-lib (#5213)", 2022-06-22).
Environment: dash 0.5.11.5-1, gcc 12.1.0-2, Artix Linux, ext4 on an HDD.
Commands used for benchmarking each commit: