-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
incompatible_allow_python_version_transitions: Python version is no longer "sticky" #7307
Comments
This makes available --incompatible_allow_python_version_transitions and --incompatible_remove_old_python_version_api, which were previously named "--experimental_...". See #7307 and #7308 respectively to track the migration for each flag. Work toward #6583, #7307, and #7308. RELNOTES[INC]: Two changes to native Python rules: 1) `default_python_version` and `--force_python` are deprecated; use `python_version` and `--python_version` respectively instead. You can preview the removal of the deprecated names with --incompatible_remove_old_python_version_api. See [#7308](#7308). 2) The version flag will no longer override the declared version of a `py_binary` or `py_test` target. You can preview this new behavior with --incompatible_allow_python_version_transitions. See [#7307](#7307). PiperOrigin-RevId: 231980615
Baseline: 441fd75 Cherry picks: + 6ca7763: Fix a typo + 2310b1c: Ignore SIGCHLD in test setup script + f9eb1b5: Complete channel initialization in the event loop Incompatible changes: - //src:bazel uses the minimal embedded JDK, if you want to avoid the extra steps of minimizing the JDK, use //src:bazel-dev instead. - //src:bazel uses the minimal embedded JDK, if you want to avoid the extra steps of building bazel with the minimized JDK, use //src:bazel-dev instead. - The default value of --host_platform and --platforms will be changed to not be dependent on the configuration. This means that setting --cpu or --host_cpu will not affect the target or host platform. - Toolchain resolution for cc rules is now enabled via an incompatible flag, --incompatible_enable_cc_toolchain_resolution. The previous flag, --enabled_toolchain_types, is deprecated and will be removed. - java_(mutable_|)proto_library: removed strict_deps attribute. - Python rules will soon reject the legacy "py" struct provider (preview by enabling --incompatible_disallow_legacy_py_provider). Upgrade rules to use PyInfo instead. See [#7298](#7298). - java_(mutable_|)proto_library: removed strict_deps attribute. - Two changes to native Python rules: 1) `default_python_version` and `--force_python` are deprecated; use `python_version` and `--python_version` respectively instead. You can preview the removal of the deprecated names with --incompatible_remove_old_python_version_api. See [#7308](#7308). 2) The version flag will no longer override the declared version of a `py_binary` or `py_test` target. You can preview this new behavior with --incompatible_allow_python_version_transitions. See [#7307](#7307). Important changes: - There is a new flag available `--experimental_java_common_create_provider_enabled_packages` that acts as a whitelist for usages of `java_common.create_provider`. The constructor will be deprecated in Bazel 0.23. - [#7024] Allow chaining of the same function type in aquery. - Introduces --local_{ram,cpu}_resources, which will take the place of --local_resources. - [#6930] Add documentation for the aquery command. - Incompatible flag `--incompatible_dont_emit_static_libgcc` has been flipped (#6825) - Incompatible flag `--incompatible_linkopts_in_user_link_flags` has been flipped (#6826) - Flag --incompatible_range_type is removed. - Flag --incompatible_disallow_slash_operator is removed. - Flag --incompatible_disallow_conflicting_providers is removed. - `--incompatible_disallow_data_transition` is now enabled by default - Allow inclusion of param files in aquery output - [#6985] Add test to verify aquery's behavior for Cpp action templates. - --incompatible_require_feature_configuration_for_pic was flipped (#7007). - Also ignore module-info.class in multi-version Jars - objc_framework has been deleted. Please refer to apple_dynamic_framework_import and apple_static_framework_import rules available in [rules_apple](https://github.com/bazelbuild/rules_apple/blob/maste r/doc/rules-general.md) - --test_sharding_strategy=experimental_heuristic is no more - objc_bundle_library has been removed. Please migrate to rules_apple's [apple_resource_bundle](https://github.com/bazelbuild/rules_apple/ blob/master/doc/rules-resources.md#apple_resource_bundle). - You can now use the attribute `aapt_version` or the flag `--android_aapt` to pick the aapt version for android_local_test tests - In --keep_going mode, Bazel now correctly returns a non-zero exit code when encountering a package loading error during target pattern parsing of patterns like "//foo:all" and "//foo/...". - The default value for --incompatible_strict_action_env has been flipped to 'false' again, as we discovered breakages for local execution users. We'll need some more time to figure out the best way to make this work for local and remote execution. Follow #7026 for more details. - Locally-executed spawns tagged "no-cache" no longer upload their outputs to the remote cache. - Introduces --host_compiler flag to allow setting a compiler for host compilation when --host_crosstool_top is specified. - --incompatible_expand_directories is enabled by default - [aquery] Handle the case of aspect-on-aspect. - Fixed a longstanding bug in the http remote cache where the value passed to --remote_timeout would be interpreted as milliseconds instead of seconds. - Enable --incompatible_use_jdk10_as_host_javabase by default, see #6661 - Add --incompatible_use_jdk11_as_host_javabase: makes JDK 11 the default --host_javabase for remote jdk (#7219) - Highlight TreeArtifact in aquery text output. - Locally-executed spawns tagged "no-cache" no longer upload their outputs to the remote cache. - java_common APIs now accept JavaToolchainInfo and JavaRuntimeInfo instead of configured targets for java_toolchain and java_runtime - cc_common.create_cc_toolchain_config_info is stable and available for production use - incompatible_use_toolchain_providers_in_java_common: pass JavaToolchainInfo and JavaRuntimeInfo providers to java_common APIs instead of configured targets (#7186) - --incompatible_strict_argument_ordering is enabled by default. - Bazel now supports reading cache hits from a repository cache, even if it doesn't have write access to the cache. - Adding arm64e to OSX CROSSTOOL. - Ignore package-level licenses on config_setting. - Add an optional output_source_jar parameter to java_common.compile - --incompatible_disable_objc_provider_resources is now enabled by default. This disables ObjcProvider's fields related to resource processing. - Explicitly set https.protocols and exclude TLSv1.3. - Bazel now validates that JAVA_HOME points to a valid JDK and falls back to auto-detection by looking up the path of `javac`. - Upgrade the embedded JDK version to 11.0.2. - Added --incompatible_disable_crosstool_file (#7320) - --incompatible_disable_objc_provider_resources is now enabled by default. This disables ObjcProvider's fields related to resource processing. - --incompatible_disable_tools_defaults_package has been flipped. - For tests that do not generate a test.xml, Bazel now uses a separate action to generate one; this results in minor differences in the generated test.xml, and makes the generation more reliable overall. - incompatible_generate_javacommon_source_jar: java_common.compile now always generates a source jar, see #5824. - New incompatible flag --incompatible_disallow_struct_provider_syntax removes the ability for rule implementation functions to return struct. Such functions should return a list of providers instead. Migration tracking: #7347 This release contains contributions from many people at Google, as well as Benjamin Peterson, Ed Schouten, erenon, George Gensure, Greg Estren, Igal Tabachnik, Ittai Zeidman, Jannis Andrija Schnitzer, John Millikin, Keith Smiley, Kelly Campbell, Max Vorobev, nicolov, Robin Nabel.
This issue was tagged as "breaking-change-0.24" but does not appear ready to be flipped in the 0.24.0 release. If this is incorrect please comment on that issue and discuss with me. |
Now targeting 0.25. |
…n_transitions Work toward #7307. RELNOTES: None PiperOrigin-RevId: 236718484
Note the updated paragraph in Migration instructions above to account for #7655. |
According to a recent run of the bazelisk CI, the only downstream failures due to |
I'm submitting the flag flip to remove the old version api now. That should give us a clean run of the bazelisk CI for this flag tomorrow. |
The bazelisk run isn't clean because closing the migration bug for At this point I think it is in practice safe to flip the flag, but in principle we're still blocked by #7808. I'd like to fix that bug first. Shouldn't be a problem for making the 0.25 cut. |
This is an extension of 09c6b7a, which turned out to be insufficient for preventing action conflicts (between unshareable actions, e.g. C++ compilation actions). That CL avoided creating a transition where it was not needed; however in cases where it is needed, we still get conflicts due to configurations that differ trivially from the top-level configuration. This CL avoids the problem by preprocessing the top-level configuration, to avoid creating configured targets with spurious config differences. Specifically, even though --force_python and --python_version admit a "null" state meaning "not set by the user, please use the default", we immediately replace this with the actual default at the time the BuildOptions is created, so no configured target sees a null value for those options. This mechanism may also be useful for other fragments in the future that need smart default values. Fixes #7808 to unblock #7307. RELNOTES: None PiperOrigin-RevId: 240207632
Submitting the flag flip because #7808 is fixed. |
Baseline: 0366246 Cherry picks: + 3f7f255: Windows: fix native test wrapper's arg. escaping + afeb8d0: Flip --incompatible_windows_escape_jvm_flags + 4299b65: Sort DirectoryNode children to ensure validity. + 231270c: Conditionally use deprecated signature for initWithContentsOfURL + 75a3a53: Add http_archive entries for testing with various JDK versions. + 4a6354a: Now that ubuntu1804 uses JDK 11, remove explicit ubuntu1804_java11 tests. + ae102fb: Fix wrong name of ubuntu1804_javabase9 task. + 0020a97: Remove @executable_path/Frameworks from rpaths + 130f86d: Download stderr/stdout to a temporary FileOutErr Incompatible changes: - (Starlark rules) The legacy "py" provider can no longer be passed to or produced by native Python rules; use [PyInfo](https://docs.bazel.build/versions/master/skylark/lib/PyIn fo.html) instead. See [#7298](#7298) for more information. - (Python rules) The `default_python_version` attribute of the `py_binary` and `py_test` rules has been renamed to `python_version`. Also, the `--force_python` flag has been renamed to `--python_version`. See [#7308](#7308) for more information. - (Python rules) The python version now changes to whatever version is specified in a `py_binary` or `py_test`'s `python_version` attribute, instead of being forced to the value set by a command line flag. You can temporarily revert this change with `--incompatible_allow_python_version_transitions=false`. See [#7307](#7307) for more information. - --incompatible_disable_third_party_license_checking` is enabled by default - Introduced --incompatible_use_python_toolchains, which supersedes --python_top/--python_path. See #7899 and #7375 for more information. - Python 3 is now the default Python version (for `py_binary` and `py_test` targets that don't specify the `python_version` attribute). Targets that are built for Python 3 will no longer have their output put in a separate `-py3` directory; instead there is now a separate `-py2` directory for Python 2 targets. See #7359 and #7593 for more information. - objc_library resource attributes are now disabled by default. Please migrate them to data instead. See #7594 for more info. - Flip --incompatible_windows_escape_jvm_flags to true. See #7486 New features: - genrules now support a $(RULEDIR) variable that resolves to the directory where the outputs of the rule are put. - Added --incompatible_windows_native_test_wrapper flag: enables using the Bash-less test wrapper on Windows. (No-op on other platforms.) Important changes: - incompatible_use_jdk11_as_host_javabase: makes JDK 11 the default --host_javabase for remote jdk (#7219) - Makes genquery somepath output deterministic. - Tristate attributes of native rules now reject True/False (use 1/0) - Rollback of "Tristate attributes of native rules now reject True/False (use 1/0)" - Tristate attributes of native rules now reject True/False (use 1/0) - Added -incompatible_do_not_split_linking_cmdline flag. See #7670 - Tristate attributes of native rules now temporarily accept True/False again - `--incompatible_disable_legacy_crosstool_fields` has been flipped (#6861) `--incompatible_disable_expand_if_all_available_in_flag_set` has been flipped (#7008) - `--incompatible_disable_legacy_crosstool_fields` has been flipped (#6861) `--incompatible_disable_expand_if_all_available_in_flag_set... RELNOTES: None. - --incompatible_no_transitive_loads is enabled by default. - Makes TreeArtifact deterministic. - --incompatible_no_transitive_loads is enabled by default. - Android NDK C++ toolchain is now configured in Starlark. This should be a backwards compatible change, but in case of bugs blame unknown commit. - `--incompatible_disable_legacy_crosstool_fields` has been flipped (#6861) `--incompatible_disable_expand_if_all_available_in_flag_set` has been flipped (#7008) - --incompatible_no_transitive_loads is enabled by default. - --incompatible_bzl_disallow_load_after_statement is enabled - Added `--incompatible_require_ctx_in_configure_features`, see #7793 for details. - Flag --incompatible_merge_genfiles_directory is flipped. This removes the directory `bazel-genfiles` in favor of `bazel-bin`. - previously deprecated flag --experimental_remote_spawn_cache was removed - `--incompatible_disallow_load_labels_to_cross_package_boundaries` is enabled by default - Fix an issue where the Android resource processor did not surface errors from aapt2 compile and link actions. - --incompatible_no_attr_license is enabled by default - `--incompatible_disable_crosstool_file` has been flipped (#7320) - A new flag `--incompatible_string_join_requires_strings` is introduced. The sequence argument of `string.join` must contain only string elements. - --incompatible_symlinked_sandbox_expands_tree_artifacts_in_runfile s_tree has been flipped - Incompatible flag `--incompatible_disable_legacy_cc_provider` has been flipped (see #7036 for details). - Don't drop the analysis cache when the same --define flag is set multiple times and the last value is the same (e.g. if the current invocation was run with "--define foo=bar" and the previous one was run with "--define foo=baz --define foo=bar"). - The --incompatible_disable_genrule_cc_toolchain_dependency flag has been flipped (see #6867 for details). - Incompatible change `--incompatible_remove_cpu_and_compiler_attributes_from_cc_toolcha in` has been flipped (see #7075 for details). - --noexperimental_java_coverage is a no-op flag. - --experimental_java_coverage/--incompatible_java_coverage flag was removed. See #7425. - incompatible_use_toolchain_providers_in_java_common: pass JavaToolchainInfo and JavaRuntimeInfo providers to java_common APIs instead of configured targets (#7186.) - --incompatible_remote_symlinks has been flipped. The remote caching and execution protocol will now represent symlinks in outputs as such. See #7917 for more details. - Bazel is now ~20MiB smaller, from unbundling the Android rules' runtime dependencies. This release contains contributions from many people at Google, as well as Andreas Herrmann, Andrew Suffield, Andy Scott, Benjamin Peterson, Ed Baunton, George Gensure, Ian McGinnis, Ity Kaul, Jingwen Chen, John Millikin, Keith Smiley, Marwan Tammam, Mike Fourie, Oscar Bonilla, perwestling, petros, Robert Sayre, Ryan Beasley, silvergasp, Stanimir Mladenov, Travis Cline, Vladimir Chebotarev, ??.
Baseline: 0366246 Cherry picks: + 3f7f255: Windows: fix native test wrapper's arg. escaping + afeb8d0: Flip --incompatible_windows_escape_jvm_flags + 4299b65: Sort DirectoryNode children to ensure validity. + 231270c: Conditionally use deprecated signature for initWithContentsOfURL + 75a3a53: Add http_archive entries for testing with various JDK versions. + 4a6354a: Now that ubuntu1804 uses JDK 11, remove explicit ubuntu1804_java11 tests. + ae102fb: Fix wrong name of ubuntu1804_javabase9 task. + 0020a97: Remove @executable_path/Frameworks from rpaths + 130f86d: Download stderr/stdout to a temporary FileOutErr Incompatible changes: - (Starlark rules) The legacy "py" provider can no longer be passed to or produced by native Python rules; use [PyInfo](https://docs.bazel.build/versions/master/skylark/lib/PyIn fo.html) instead. See [#7298](#7298) for more information. - (Python rules) The `default_python_version` attribute of the `py_binary` and `py_test` rules has been renamed to `python_version`. Also, the `--force_python` flag has been renamed to `--python_version`. See [#7308](#7308) for more information. - (Python rules) The python version now changes to whatever version is specified in a `py_binary` or `py_test`'s `python_version` attribute, instead of being forced to the value set by a command line flag. You can temporarily revert this change with `--incompatible_allow_python_version_transitions=false`. See [#7307](#7307) for more information. - --incompatible_disable_third_party_license_checking` is enabled by default - Introduced --incompatible_use_python_toolchains, which supersedes --python_top/--python_path. See #7899 and #7375 for more information. - Python 3 is now the default Python version (for `py_binary` and `py_test` targets that don't specify the `python_version` attribute). Targets that are built for Python 3 will no longer have their output put in a separate `-py3` directory; instead there is now a separate `-py2` directory for Python 2 targets. See #7359 and #7593 for more information. - objc_library resource attributes are now disabled by default. Please migrate them to data instead. See #7594 for more info. - Flip --incompatible_windows_escape_jvm_flags to true. See #7486 New features: - genrules now support a $(RULEDIR) variable that resolves to the directory where the outputs of the rule are put. - Added --incompatible_windows_native_test_wrapper flag: enables using the Bash-less test wrapper on Windows. (No-op on other platforms.) Important changes: - incompatible_use_jdk11_as_host_javabase: makes JDK 11 the default --host_javabase for remote jdk (#7219) - Makes genquery somepath output deterministic. - Tristate attributes of native rules now reject True/False (use 1/0) - Rollback of "Tristate attributes of native rules now reject True/False (use 1/0)" - Tristate attributes of native rules now reject True/False (use 1/0) - Added -incompatible_do_not_split_linking_cmdline flag. See #7670 - Tristate attributes of native rules now temporarily accept True/False again - `--incompatible_disable_legacy_crosstool_fields` has been flipped (#6861) `--incompatible_disable_expand_if_all_available_in_flag_set` has been flipped (#7008) - `--incompatible_disable_legacy_crosstool_fields` has been flipped (#6861) `--incompatible_disable_expand_if_all_available_in_flag_set... RELNOTES: None. - --incompatible_no_transitive_loads is enabled by default. - Makes TreeArtifact deterministic. - --incompatible_no_transitive_loads is enabled by default. - Android NDK C++ toolchain is now configured in Starlark. This should be a backwards compatible change, but in case of bugs blame unknown commit. - `--incompatible_disable_legacy_crosstool_fields` has been flipped (#6861) `--incompatible_disable_expand_if_all_available_in_flag_set` has been flipped (#7008) - --incompatible_no_transitive_loads is enabled by default. - --incompatible_bzl_disallow_load_after_statement is enabled - Added `--incompatible_require_ctx_in_configure_features`, see #7793 for details. - Flag --incompatible_merge_genfiles_directory is flipped. This removes the directory `bazel-genfiles` in favor of `bazel-bin`. - previously deprecated flag --experimental_remote_spawn_cache was removed - `--incompatible_disallow_load_labels_to_cross_package_boundaries` is enabled by default - Fix an issue where the Android resource processor did not surface errors from aapt2 compile and link actions. - --incompatible_no_attr_license is enabled by default - `--incompatible_disable_crosstool_file` has been flipped (#7320) - A new flag `--incompatible_string_join_requires_strings` is introduced. The sequence argument of `string.join` must contain only string elements. - --incompatible_symlinked_sandbox_expands_tree_artifacts_in_runfile s_tree has been flipped - Incompatible flag `--incompatible_disable_legacy_cc_provider` has been flipped (see #7036 for details). - Don't drop the analysis cache when the same --define flag is set multiple times and the last value is the same (e.g. if the current invocation was run with "--define foo=bar" and the previous one was run with "--define foo=baz --define foo=bar"). - The --incompatible_disable_genrule_cc_toolchain_dependency flag has been flipped (see #6867 for details). - Incompatible change `--incompatible_remove_cpu_and_compiler_attributes_from_cc_toolcha in` has been flipped (see #7075 for details). - --noexperimental_java_coverage is a no-op flag. - --experimental_java_coverage/--incompatible_java_coverage flag was removed. See #7425. - incompatible_use_toolchain_providers_in_java_common: pass JavaToolchainInfo and JavaRuntimeInfo providers to java_common APIs instead of configured targets (#7186.) - --incompatible_remote_symlinks has been flipped. The remote caching and execution protocol will now represent symlinks in outputs as such. See #7917 for more details. - Bazel is now ~20MiB smaller, from unbundling the Android rules' runtime dependencies. This release contains contributions from many people at Google, as well as Andreas Herrmann, Andrew Suffield, Andy Scott, Benjamin Peterson, Ed Baunton, George Gensure, Ian McGinnis, Ity Kaul, Jingwen Chen, John Millikin, Keith Smiley, Marwan Tammam, Mike Fourie, Oscar Bonilla, perwestling, petros, Robert Sayre, Ryan Beasley, silvergasp, Stanimir Mladenov, Travis Cline, Vladimir Chebotarev, ??.
With this flag enabled by default, the Python version is no longer "sticky" and forced to a fixed value by a command line flag. Instead it adjusts to whatever the `py_binary` or `py_test` declares its version to be. For more information see feature tracking issue #6583 and flag migration tracking issue #7307. Closes #7307. RELNOTES[INC]: (Python rules) The python version now changes to whatever version is specified in a `py_binary` or `py_test`'s `python_version` attribute, instead of being forced to the value set by a command line flag. You can temporarily revert this change with `--incompatible_allow_python_version_transitions=false`. See [#7307](bazelbuild/bazel#7307) for more information. PiperOrigin-RevId: 240227319
Flag:
--incompatible_allow_python_version_transitions
Available since: 0.23
Will be flipped in: either 0.24 or 0.25, depending on migration progress
Tracking issue: #6583
Design doc
Corresponding syntactic change: #7308
Motivation
The Python mode behaved in a confusing and error-prone way, and didn't support important use cases like a Python 3 binary depending on a Python 2 one in its
data
attribute.See the design doc for more details.
Note: This incompatible change affects the mechanism that Bazel uses to decide whether a target should be built for Python 2 or 3. However, due to #4815, even when a target is built for Python 3 it may still execute under a Python 2 interpreter. See the workaround using
py_runtime
/--python_top
in #4815 (comment).Change
This makes the Python version have only two possible states:
PY2
andPY3
. The version can change whenever an executable Python rule (py_binary
orpy_test
) is encountered, even if the version was previously set by another target or by a top-level flag.In addition,
srcs_version
validation is done at the level of the executable target, rather than in eachpy_library
. This makes it so you can dobazel build //...
without getting errors from PY2-only or PY3-only libraries.This change does not remove any syntax. See
--incompatible_remove_old_python_version_api
for that.Note: This change does not affect targets built in the
host
configuration, such as targets ingenrule
'stools
attribute, or user-defined rules that setcfg = host
on an attribute. This is a consequence of technical limitations on the host configuration to be addressed in future work. In the meantime, you can set a Python version to use globally for all host-configured targets by setting--host_force_python=[PY2 | PY3]
.Migration
If you were relying on
--force_python
to control your Python mode, that won't work anymore sincepy_binary
will set its own mode. Use thepython_version
attribute (formerlydefault_python_version
) onpy_binary
instead.Using multiple Python versions in the same build can lead to action conflicts if you test the python version by
select()
-ing on"force_python"
. To avoid this, you should instead select on@bazel_tools//tools/python:python_version
. See here. You can ensure that your build doesn't select on"force_python"
by enabling--incompatible_remove_old_python_version_api
.Another type of action conflict can arise due to #7655 when a
cc_library
is data-depended on by apy_binary
. This kind of conflict can be avoided by enabling--incompatible_remove_old_python_version_api
. If you're not able to migrate your build to enable that flag yet, you can also avoid this particular conflict by explicitly passing a value for the--force_python
flag that matches the value passed for--python_version
(or the default value if--python_version
is not passed).If there's a
srcs_version
conflict in your build, you will no longer see an error at thepy_library
that introduces the conflict. Instead you'll see it in thepy_binary
orpy_test
that transitively depends on the library. The error message will not identify which library introduced the version constraint. To find it, you can run the aspect@bazel_tools//tools/python:srcs_version.bzl%find_requirements
as follows:This will build a
[...]-pyversioninfo.txt
file explaining what dependency requires what Python version. You can run it directly on thepy_binary
orpy_test
that failed due to a version conflict.The text was updated successfully, but these errors were encountered: