Disable GNU `make` implicit variables and Quiet `ar` with `ARFLAGS` rather than `> null` #2711

ParadoxV5 · 2024-04-18T00:13:21Z

CC, AR and ARFLAGS (and also RM and CXX) are all implicit variables in GNU make.

CC and AR already defaults to cc and ar, which means CC ?= cc does nothing since before I started tinkering.
ARFLAGS defaults to -rv, where v stands for verbose.
- https://stackoverflow.com/a/40516532 hints that changing it to r avoids printing to null,
  and printing to /dev/null was the final obstacle for me to build nightly Prism on my MSYS-less Windows setup!

Rather than adapting the Makefile and extconf.rb around them, I disabled them with --no-builtin-variables. While this flag may be GNU-exclusive, the README implies that we only support GNU make.

CURDIR from Avoid shelling *nix commands #2706 is also possibly GNU-exclusive.
The popularity of GNU means it’s a-okay to be toolset-specific rather than platform-specific.

Previously conflicted with Update makefile to use / fall back on built-in "ar" #2716

ParadoxV5 · 2024-04-18T00:20:57Z

Makefile

@@ -12,7 +12,7 @@ SOEXT ?= $(shell ruby -e 'puts RbConfig::CONFIG["SOEXT"]')

 CPPFLAGS := -Iinclude $(CPPFLAGS)
 CFLAGS := -g -O2 -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden $(CFLAGS)
-CC ?= cc
+ARFLAGS := -r


I introduced ?= in #2705 to be explicit that the variable may be overwritten by env vars sent from extconf.rb.
But a quick test shows that <implicit variable> ?= <anything> doesn’t do anything because the variable was already set – implicitly by GNU make! 😅
This is why this assignment must be either = or := but not ?=. It shouldn’t affect receiving the rbconfig ARFLAGS through the aforementioned env vars.

TODO: circument only for ar -rv; do not change for libtool (such as used in that one failing CI)

Switched to --no-builtin-variables. Explicit is better than implicit anyways.

ParadoxV5 · 2024-04-18T00:24:09Z

Makefile

@@ -36,7 +36,7 @@ build/libprism.$(SOEXT): $(SHARED_OBJECTS)

 build/libprism.a: $(STATIC_OBJECTS)
 	$(ECHO) "building $@"
-	$(Q) $(AR) $(ARFLAGS) $@ $(STATIC_OBJECTS) $(Q1:0=>/dev/null)
+	$(Q) $(AR) $(ARFLAGS) $@ $(STATIC_OBJECTS)


A quick text find shows that this was the only recipe where Q1 is used, and the only other use of Q1 is

prism/Makefile

Line 6 in 17762fa

Q = $(Q1:0=@)

, which may now be flattenend.
(I’m a novice in C-land; if the entire set of Vs and Qs is a convention, introduce them to me.)

For this matter, ba82ce9 introduces an acutal use for V0:

prism/Makefile

Line 17 in ba82ce9

ARFLAGS ?= -r$(V0:1=v)

ParadoxV5 · 2024-04-18T00:45:03Z

(Mac Ruby 2.7 CI failure)

Intriguing.
CC and co. should be set from the rbconfig in practice. Considering this was never an issue in my previous Makefile PRs, does this mean that the Mac Ruby 2.7 was set with libtool for AR, nothing for ARFLAGS, and make had a different default for Mac?
I do not know how to investigate a Mac problem.

Easy solution: drop Ruby 2.7 support (we’re still in 0.x aren’t we?)

kddnewton · 2024-04-19T18:26:50Z

Unfortunately we cannot drop 2.7 support, as that is BASERUBY and we need it to build Ruby itself.

kddnewton · 2024-04-19T18:28:17Z

CC and AR already defaults to cc and ar, so setting CC to cc does nothing since before I started tinkering.

I'm not sure this is true. Before, we always set CC to cc, so it couldn't be overridden by env vars. Now it's going to be the same CC that Ruby is compiled with.

ParadoxV5 · 2024-04-19T20:55:15Z

Before, we always set CC to cc, so it couldn't be overridden by env vars.

Like with ARFLAGS in the PR, I expect the original CC := cc to be overridable by env vars.

Now it's going to be the same CC that Ruby is compiled with.

This though is intended.

ParadoxV5 · 2024-04-19T23:29:21Z

Like with ARFLAGS in the PR, I expect the original CC := cc to be overridable by env vars.

Oof, I had it mixed up. It’s only overridable with make KEY=VALUE, not from env vars.

kddnewton · 2024-05-24T16:49:34Z

Hey @ParadoxV5 are you still working on this or should this be closed?

ParadoxV5 · 2024-05-24T17:52:52Z

Hey @ParadoxV5 are you still working on this or should this be closed?

Hey, @kddnewton, now that https://bugs.ruby-lang.org/issues/20499 has properly fixed #2716 in the Ruby upstream, I plan to get back to this these few of days.
By the way, any plans to revert the workaround #2725 (after Ruby releases teenies with the fix)? [c.c. @eregon]

kddnewton · 2024-05-24T20:12:38Z

I think I'd prefer to leave it because I'd like to support all of the Ruby versions in the range that we support, which includes patch revisions.

eregon · 2024-05-25T02:29:44Z

Yeah agreed let's keep that workaround for existing releases and link to the relevant issues to make it easy to figure out from the code why that is needed

`ARFLAGS` is an implicit variable in GNU `make` (the `README` _specifies GNU_ `make`) and defaulted verbosely, so un-verbose-fy it is the more direct *and platform-agnostic* way to quiet `ar`.

As observed in ruby#2771, `?=` thinks that implicit definitions of CC`, `AR` and `ARFLAGS` are explicit. Disabling implicit variables is more straightforward than adapting the `Makefile` and `extconf.rb` around them.

`File::exist?` does not check the PATH, meaning the checks fail as long as `CC`/`AR` aren’t absolute paths. OTOH, `MakeMakefile#find_executable` does.

kddnewton · 2024-05-28T15:29:49Z

ext/prism/extconf.rb

@@ -51,6 +51,7 @@ def make(env, target)
    system(
      env,
      RUBY_PLATFORM.include?("openbsd") ? "gmake" : "make",
+      "--no-builtin-variables", # don't let GNU make implicit variables override variable fallbacks in the Makefile


I'm not sure I understand what this is doing. Could you explain why we need this?

As noted on the previous posts, this flag disables implicit variables for GNU make (updated top post).
GNU make takes those implicit variables precedence over our ?=s, especially the new ARFLAGS ?= -r$(V0:1=v) (PR thread), meaning they aren’t applied without this flag.

prism/Makefile

Lines 15 to 17 in 9bb8710

CC ?= cc

AR ?= ar

ARFLAGS ?= -r$(V0:1=v)

kddnewton · 2024-12-12T01:16:21Z

@ParadoxV5 now that the build system has changed because it's not shelling out to make anymore, can you check if this is still necessary? If it is can you rebase?

ParadoxV5

now that the build system has changed because it's not shelling out to make anymore, can you check if this is still necessary?

Interesting, @kddnewton.

At #3273, @mame was practically requesting Prism-gem support for MSVC, and it was approved by the decision to stop intercepting the platform-agnostic mkmf.
FWIW, MSVC is also not MSYS.

My goal was to support MSYS (DevKit in RubyInstaller’s jargon)-less MinGW.
I built the latest HEAD and can confirm that, because the RbConfig-based mkmk absorbs all setup differences, that decision covers my case as well and it no longer requires this PR.

The only purpose left in this patch, then, is to improve the general code quality.

ParadoxV5 · 2024-12-14T23:57:24Z

Makefile

@@ -37,7 +38,7 @@ build/libprism.$(SOEXT): $(SHARED_OBJECTS)

 build/libprism.a: $(STATIC_OBJECTS)
 	$(ECHO) "building $@ with $(AR)"
-	$(Q) $(AR) $(ARFLAGS) $@ $(STATIC_OBJECTS) $(Q1:0=>/dev/null)
+	$(Q) $(AR) $(ARFLAGS) $@ $(STATIC_OBJECTS)


Writing to Null has always been questionable as we could simply not supply -v to ar.
As stated in OP, this workaround exists because ARFLAGS defaults to -rv, unless forcefully replaced.

Alternate approaches include:

use RbConfig::CONFIG for those (i.e., all rbconfig.rb cases) as well

tricking create_makefile also counts

Do those still need a separate .a and ar steps?
Most C projects (mkmf or not) builds .cs to .os and .os to .so/.dll; there’s no .a step after .os.

Follow-up of Fails to install gem on mswin64 #3273 discussion

ParadoxV5 · 2024-12-15T00:38:30Z

Makefile

 CC ?= cc
 AR ?= ar
+ARFLAGS ?= -r$(V0:1=v)


According to #2716, the original ?=s were once a workaround for #2716.
That issue’s root cause’s separately fixed and 7c2461f purged its related extconf.rb code. Like this PR, I wonder if those lines are still necessary?

ParadoxV5 commented Apr 18, 2024

View reviewed changes

ParadoxV5 mentioned this pull request Apr 19, 2024

Update makefile to use / fall back on built-in "ar" #2716

Closed

ParadoxV5 marked this pull request as draft April 19, 2024 23:18

ParadoxV5 mentioned this pull request Apr 22, 2024

Fall back to default AR and CC #2725

Merged

ParadoxV5 added 2 commits May 25, 2024 14:22

Quiet ar with ARFLAGS rather than > null

ba82ce9

`ARFLAGS` is an implicit variable in GNU `make` (the `README` _specifies GNU_ `make`) and defaulted verbosely, so un-verbose-fy it is the more direct *and platform-agnostic* way to quiet `ar`.

make with --no-builtin-variables

0779fa9

As observed in ruby#2771, `?=` thinks that implicit definitions of CC`, `AR` and `ARFLAGS` are explicit. Disabling implicit variables is more straightforward than adapting the `Makefile` and `extconf.rb` around them.

ParadoxV5 force-pushed the gnu-make-implicit-vars branch from 17762fa to 0779fa9 Compare May 25, 2024 20:46

ParadoxV5 marked this pull request as ready for review May 25, 2024 21:00

ParadoxV5 changed the title ~~Quiet ar with ARFLAGS rather than > null & Remove CC ?= cc~~ Disable GNU make implicit variables and Quiet ar with ARFLAGS rather than > null May 25, 2024

ParadoxV5 marked this pull request as draft May 25, 2024 21:57

Search the PATH for CC/AR checks

9bb8710

`File::exist?` does not check the PATH, meaning the checks fail as long as `CC`/`AR` aren’t absolute paths. OTOH, `MakeMakefile#find_executable` does.

ParadoxV5 marked this pull request as ready for review May 25, 2024 22:19

kddnewton reviewed May 28, 2024

View reviewed changes

ParadoxV5 added 2 commits June 14, 2024 21:28

Merge branch 'ruby:main' into gnu-make-implicit-vars

7ce2a66

Merge branch 'main' into gnu-make-implicit-vars

469487f

ParadoxV5 commented Dec 15, 2024

View reviewed changes

ParadoxV5 marked this pull request as draft December 15, 2024 00:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable GNU `make` implicit variables and Quiet `ar` with `ARFLAGS` rather than `> null` #2711

Disable GNU `make` implicit variables and Quiet `ar` with `ARFLAGS` rather than `> null` #2711

ParadoxV5 commented Apr 18, 2024 •

edited

Loading

ParadoxV5 Apr 18, 2024

ParadoxV5 Apr 19, 2024

ParadoxV5 May 25, 2024

ParadoxV5 Apr 18, 2024 •

edited

Loading

ParadoxV5 May 25, 2024

ParadoxV5 commented Apr 18, 2024

kddnewton commented Apr 19, 2024

kddnewton commented Apr 19, 2024

ParadoxV5 commented Apr 19, 2024

ParadoxV5 commented Apr 19, 2024

kddnewton commented May 24, 2024

ParadoxV5 commented May 24, 2024

kddnewton commented May 24, 2024

eregon commented May 25, 2024

kddnewton May 28, 2024

ParadoxV5 May 28, 2024 •

edited

Loading

kddnewton commented Dec 12, 2024

ParadoxV5 left a comment •

edited

Loading

ParadoxV5 Dec 14, 2024

ParadoxV5 Dec 15, 2024

Disable GNU make implicit variables and Quiet ar with ARFLAGS rather than > null #2711

Are you sure you want to change the base?

Disable GNU make implicit variables and Quiet ar with ARFLAGS rather than > null #2711

Conversation

ParadoxV5 commented Apr 18, 2024 • edited Loading

ParadoxV5 Apr 18, 2024

Choose a reason for hiding this comment

ParadoxV5 Apr 19, 2024

Choose a reason for hiding this comment

ParadoxV5 May 25, 2024

Choose a reason for hiding this comment

ParadoxV5 Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

ParadoxV5 May 25, 2024

Choose a reason for hiding this comment

ParadoxV5 commented Apr 18, 2024

kddnewton commented Apr 19, 2024

kddnewton commented Apr 19, 2024

ParadoxV5 commented Apr 19, 2024

ParadoxV5 commented Apr 19, 2024

kddnewton commented May 24, 2024

ParadoxV5 commented May 24, 2024

kddnewton commented May 24, 2024

eregon commented May 25, 2024

kddnewton May 28, 2024

Choose a reason for hiding this comment

ParadoxV5 May 28, 2024 • edited Loading

Choose a reason for hiding this comment

kddnewton commented Dec 12, 2024

ParadoxV5 left a comment • edited Loading

Choose a reason for hiding this comment

ParadoxV5 Dec 14, 2024

Choose a reason for hiding this comment

ParadoxV5 Dec 15, 2024

Choose a reason for hiding this comment

Disable GNU `make` implicit variables and Quiet `ar` with `ARFLAGS` rather than `> null` #2711

Disable GNU `make` implicit variables and Quiet `ar` with `ARFLAGS` rather than `> null` #2711

ParadoxV5 commented Apr 18, 2024 •

edited

Loading

ParadoxV5 Apr 18, 2024 •

edited

Loading

ParadoxV5 May 28, 2024 •

edited

Loading

ParadoxV5 left a comment •

edited

Loading