Add --incompatible_merge_fixed_and_default_shell_env

[fmeum]

With the new flag, Starlark actions that specify both env and use_default_shell_env will no longer have env ignored. Instead, the values of env will override the default shell environment.

This allows Starlark actions to both pick up user-configured variables such as PATH from the shell environment as well as set variables to fixed values required for the action, e.g., variables provided by the C++ toolchain.

Rationale for having env override the default shell env: The rules know best which values they have to set specific environment variables to in order to successfully execute an action, so a situation where users break an action by a globally applied --action_env is prevented. If users really do have to be able to modify an environment variables fixed by the rule, the rule can always make this configurable via an attribute.

Work towards #5980
Fixes #12049

[fmeum]

@comius I thought more about #5980 and found this to be the only approach that made sense to me, so I went ahead and created a PR. Let me know if you think this deserves more design work.

[fmeum]

Cc @oquenchil as this fixes an issue with cc_import

[gregestren]

I think @comius could best triage?

[fmeum]

Learned from @keith that this may end up fixing #12049.

[keith]

bump, this would be super nice to have

[fmeum]

@comius It would be great if you could take a look or reassign to someone else. Flipping this flag is required for my ongoing work on the all Starlark cc_static_library rule.

[keith]

would definitely still love to see this one land!

[fmeum]

@comius Friendly ping, could you take a look or assign to someone else?

[brentleyjones]

@bazelbuild/triage

[iancha1992]

@bazelbuild/triage

@comius will try and review this tmrw

cc: @fmeum @brentleyjones

[brentleyjones]

Since this is behind a flag, you think it's cherry-pickable @fmeum?

[fmeum]

Since this is behind a flag, you think it's cherry-pickable @fmeum?

It would be if it applies cleanly. I recall some merge conflicts, but we can give it a try.

[fmeum]

@bazel-io flag

[iancha1992]

@bazel-io fork 6.4.0

[iancha1992]

@fmeum @brentleyjones @keith @gregestren @comius
I think we may need another commit(s) before we can cherry-pick this one.

From src/main/java/com/google/devtools/build/lib/analysis/actions/SpawnAction.java

In the current release-6.4.0 branch, we have

ActionEnvironment env =
          actionEnvironment != null
              ? actionEnvironment
              : useDefaultShellEnvironment
                  ? configuration.getActionEnvironment()
                  : ActionEnvironment.create(environment, inheritedEnvironment);
      return buildSpawnAction(
          owner, commandLines, configuration.getCommandLineLimits(), configuration, env);

but it really should be

ActionEnvironment env =
          useDefaultShellEnvironment
              ? configuration.getActionEnvironment()
              : ActionEnvironment.create(environment);

before the change.

cc: @bazelbuild/triage

[fmeum]

I submitted a manual cherry-pick as #19319.