Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make AndroidTests pass with incompatible_disallow_empty_glob #15374

Closed
wants to merge 1 commit into from

Conversation

limdor
Copy link
Contributor

@limdor limdor commented Apr 29, 2022

There are several globs that are empty and this prevents building
with the incompatible flag #8195.
This commit just makes it explicit that empty is allowed.

@limdor limdor requested review from ahumesky and ted-xie as code owners April 29, 2022 19:43
@ahumesky
Copy link
Contributor

So were each of these globs actually failing with --incompatible_disallow_empty_glob? Checking a few of these for example, the globs in android_ndk_misc_libraries_template.txt, the cpufeatures files are still in the NDK, so those globs should match something

@limdor
Copy link
Contributor Author

limdor commented Apr 29, 2022

How I came up with these globs and not others?
These allow_empty True were needed to flip the flag #15327
The same for #15339 and #15330
I splitted in 3 to make it easier but I can also put them together or split them per folder.
Just let me know what would be the preference.

Once these 3 are merged, the flag still cannot be flipped because of protocolbuffers/upb#584

@ahumesky
Copy link
Contributor

So adding allow_empty = True would move us closer to enabling the flag, but it would seem to be better to investigate why each of these globs is empty. And again in the example of cpufeatures, the NDK still has for example sources/android/cpufeatures/cpu-features.c so that glob shouldn't be returning nothing -- there might actually be something wrong with the glob or something else.

Is there someone on the bazel team who you're working with to drive this effort?

At minimum it would be good to create a separate issue to attach as a TODO to each glob that has allow_empty = True added to it so that we know why the code is the way that it is.

@limdor
Copy link
Contributor Author

limdor commented Apr 29, 2022

Is there someone on the bazel team who you're working with to drive this effort?

No, who would be the right person?

And again in the example of cpufeatures, the NDK still has for example sources/android/cpufeatures/cpu-features.c so that glob shouldn't be returning nothing -- there might actually be something wrong with the glob or something else.

The problem is not with the example but when running the test //src/test/java/com/google/devtools/build/lib/bazel/rules/android:AndroidTests

At minimum it would be good to create a separate issue to attach as a TODO to each glob that has allow_empty = True added to it so that we know why the code is the way that it is.

If you prefer I can do that

One of the reasons why is not possible to flip
incompatible_disallow_empty_glob is because AndroidTests are failing

//src/test/java/com/google/devtools/build/lib/bazel/rules/android:AndroidTests

This commit add allow_empty = True to reflect the default behavior and
make the tests pass.
@limdor limdor changed the title Be explicit about alowing empty globs (third part) Make AndroidTests pass with incompatible_disallow_empty_glob Apr 29, 2022
@limdor
Copy link
Contributor Author

limdor commented Apr 29, 2022

@ahumesky I've updated the changes to include only the changes strictly needed for AndroidTests. If you think that this is not ok then probably the tests should be adapted instead.

@ckolli5 ckolli5 added the team-Android Issues for Android team label Apr 29, 2022
@@ -186,7 +186,7 @@ public void setupMockClient(MockToolsConfig config, List<String> workspaceConten
"java_host_runtime_alias(name = 'current_host_java_runtime')",
"filegroup(name='langtools', srcs=['jdk/lib/tools.jar'])",
"filegroup(name='bootclasspath', srcs=['jdk/jre/lib/rt.jar'])",
"filegroup(name='extdir', srcs=glob(['jdk/jre/lib/ext/*']))",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment about this. It does not look like android specific but it is needed also to make the android tests work. Maybe someone else apart from the android team should look at it.

@limdor
Copy link
Contributor Author

limdor commented Jul 23, 2022

@ahumesky @ted-xie friendly reminder.
Could you give me an update on what would be needed in your opinion in order to move forward? Thanks

@sgowroji sgowroji added the awaiting-review PR is awaiting review from an assigned reviewer label Aug 24, 2022
@limdor limdor mentioned this pull request Aug 24, 2022
56 tasks
@limdor
Copy link
Contributor Author

limdor commented Sep 27, 2022

Gentle reminder

@aiuto aiuto assigned sgowroji and ted-xie and unassigned sgowroji Sep 29, 2022
@aiuto aiuto self-requested a review September 29, 2022 04:01
@@ -1,6 +1,6 @@
cc_library(
name = "cpufeatures",
srcs = glob(["ndk/sources/android/cpufeatures/*.c"]),
hdrs = glob(["ndk/sources/android/cpufeatures/*.h"]),
srcs = glob(["ndk/sources/android/cpufeatures/*.c"], allow_empty = True),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't make sense to me that these could ever result in failed globs in real life, but maybe they do in our tests.
OTOH, I don't think it is a problem that we hide the glob failure. We will simply push the error from glob time until link time, but the build will still fail, so all is good.

The same question applies everywhere.

Copy link
Contributor

@aiuto aiuto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'm no nothing about these rules.

@ted-xie
Copy link
Contributor

ted-xie commented Sep 29, 2022

It also seems strange to me that any of these globs would return empty. I would expect downstream failures if various NDK features are missing source files. I'm OK with pushing this in upstream, but we should all be aware that something else entirely may be broken that's causing the source files not to show up.

aiuto pushed a commit to aiuto/bazel that referenced this pull request Oct 12, 2022
There are several globs that are empty and this prevents building
with the incompatible flag bazelbuild#8195.
This commit just makes it explicit that empty is allowed.

Closes bazelbuild#15374.

PiperOrigin-RevId: 477730831
Change-Id: I7063906f8eca5db4653fb126f4b781dec2c035cd
copybara-service bot pushed a commit that referenced this pull request Oct 7, 2024
This PR is to flip the default value of `incompatible_disallow_empty_glob`. This flag is tracked in #8195 and it is available since Bazel 0.29 from August 2019.
Once this is merged, users still can allow empty globs without being explicit setting `--incompatible_disallow_empty_glob=false`.

The motivation of the flip is bazel-contrib/SIG-rules-authors#37 and the fact of not keeping an incompatible flag for 4 years without flipping it.

What needs to be done before flipping this:

- [x] Merge protocolbuffers/upb#584
- [x] Update upb to a version that has the change in protocolbuffers/upb#584
- [x] Merge protocolbuffers/upb#745
- [x] Bring the changes from protocolbuffers/upb#745 to the 21.x branch
- [x] Bring the changes from protocolbuffers/upb@e5f2601 to the 21.x branch (protocolbuffers/upb#781)
- [x] Update upb to the latest commit of the 21.x branch (#16343)
- [x] Merge #15374
- [x] Merge #15339
- [x] Merge #15330
- [x] Merge #16431
- [x] Merge #16468
- [x] Get green checks for this PR
- [ ] Fix failures in downstream projects: https://buildkite.com/bazel/bazelisk-plus-incompatible-flags
  - [ ] Fix empty globs from IntellIJ Plugin (https://github.com/bazelbuild/intellij)
    - [ ] Issue: bazelbuild/intellij#4040
    - [ ] Android Studio Plugin
    - [ ] Android Studio Plugin Google
    - [ ] CLion Plugin
    - [ ] CLion Plugin Google
    - [ ] IntelliJ Plugin
    - [ ] IntelliJ Plugin Aspect
    - [ ] IntelliJ Plugin Aspect Google
    - [ ] IntelliJ Plugin Google
    - [ ] IntelliJ UE Plugin
    - [ ] IntelliJ UE Plugin Google
    - Opened PRs
      - [x] bazelbuild/intellij#4025
      - [x] bazelbuild/intellij#4038
  - [x] Fix empty globs from Bazel Bench (https://github.com/bazelbuild/bazel-bench)
    - [ ] bazelbuild/bazel-bench#174
    - [x] bazelbuild/bazel-bench#149
  - [x] Fix empty globs from Buildfarm (https://github.com/bazelbuild/bazel-buildfarm)
  - [x] Fix empty globs from Cargo
  - [x] ~~Fix empty globs from Cartographer (https://github.com/cartographer-project/cartographer)~~ (No longer maintained)
    - Issue: cartographer-project/cartographer#1908
    - PR to enforce it cartographer-project/cartographer#1944
  - [x] Fix empty globs from Cloud Robotics Core (https://github.com/googlecloudrobotics/core/)
    - [x] PR to prevent regresions googlecloudrobotics/core#279
  - [ ] Fix empty globs from Envoy
  - [x] Fix empty globs from FlatBuffers
  - [x] Fix empty globs from Flogger
  - [ ] Fix empty globs from Gerrit (https://gerrit.googlesource.com/gerrit.git)
    - Issue: https://issues.gerritcodereview.com/issues/40011754
  - [x] Fix empty globs from :bazel:Protobuf
  - [x] Fix empty globs from rules_android_ndk
  - [ ] Fix empty globs from rules_dotnet
  - [x] Fix empty globs from :bazel:rules_foreign_cc bazel-contrib/rules_foreign_cc#974
  - [x] Fix empty globs from rules_go
  - [x] Fix empty globs from rules_haskell tweag/rules_haskell#1827
  - [x] Fix empty globs from :bazel:rules_jvm_external
  - [x] Fix empty globs from rules_kotlin
  - [x] Fix empty globs from rules_nodejs
  - [x] Fix empty globs from :bazel:rules_python
  - [ ] Fix empty globs from rules_rust
  - [x] Fix empty globs from rules_swift
  - [x] Fix empty globs from rules_webtesting
  - [x] Fix empty globs from TensorFlow  tensorflow/tensorflow#58155
    - [x] tensorflow/tensorflow#58008
    - [x] tensorflow/tensorflow#58154
  - [x] Fix empty globs from upb

RELNOTES[INC]: The flag `--incompatible_disallow_empty_glob` now defaults to true.

Closes #15327.

PiperOrigin-RevId: 683316885
Change-Id: Id34ffd32a9e979fada9207e475b826a508100ba5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-review PR is awaiting review from an assigned reviewer team-Android Issues for Android team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants