Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: removing root src not used by Maven #1156

Merged
merged 6 commits into from
Dec 19, 2022
Merged

chore: removing root src not used by Maven #1156

merged 6 commits into from
Dec 19, 2022

Conversation

suztomo
Copy link
Member

@suztomo suztomo commented Dec 16, 2022 •
edited
Loading

The correct source of "src" of gapic-generator-java is now gapic-generator-java/src, not the one at the root.

@suztomo suztomo requested review from a team as code owners December 16, 2022 22:51
@snippet-bot
Copy link

snippet-bot bot commented Dec 16, 2022 •
edited
Loading

Here is the summary of changes.

You are about to delete 125 region tags.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@suztomo
Copy link
Member Author

suztomo commented Dec 16, 2022

"ci / build (11)" failed:

Run bazel --batch test //:units --noshow_progress --test_output=errors
2022/12/16 22:52:03 Downloading https://releases.bazel.build/5.4.0/release/bazel-5.4.0-linux-x86_64...
Extracting Bazel installation...
(22:52:0[7](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:8)) INFO: Current date is 2022-12-16
(22:52:09) DEBUG: Rule 'com_google_api_gax_java' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "5[8](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:9)[9](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:10)64aced9540c9a01d9eee9a3fa9ce0a91371894fdcd5b4e54d284f0bfe7995"
(22:52:09) DEBUG: Repository com_google_api_gax_java instantiated at:
  /home/runner/work/gapic-generator-java/gapic-generator-java/WORKSPACE:38:13: in <toplevel>
Repository rule http_archive defined at:
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
(22:52:[11](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:12)) DEBUG: Rule 'com_google_protobuf' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "8242327e5df8c80ba49e4165250b8f79a76bd11765facefaaecfca7747dc8da2"
(22:52:11) DEBUG: Repository com_google_protobuf instantiated at:
  /home/runner/work/gapic-generator-java/gapic-generator-java/WORKSPACE:53:37: in <toplevel>
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/com_google_api_gax_java/repositories.bzl:51:11: in com_google_api_gax_java_repositories
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/com_google_api_gax_java/repositories.bzl:139:14: in _maybe
Repository rule http_archive defined at:
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
(22:52:[12](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:13)) DEBUG: Rule 'com_google_googleapis' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "a8ec[13](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:14)951748bdc0ad794d03ccc34ace8e0c498babaa25e42576198430feb628"
(22:52:12) DEBUG: Repository com_google_googleapis instantiated at:
  /home/runner/work/gapic-generator-java/gapic-generator-java/WORKSPACE:57:13: in <toplevel>
Repository rule http_archive defined at:
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
(22:52:13) DEBUG: Rule 'io_grpc_grpc_java' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "f2a7ab5[14](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:15)8a1f6b451db13c5eea1185341c681c031dc775a973775e1c5d1afcb"
(22:52:13) DEBUG: Repository io_grpc_grpc_java instantiated at:
  /home/runner/work/gapic-generator-java/gapic-generator-java/WORKSPACE:53:37: in <toplevel>
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/com_google_api_gax_java/repositories.bzl:60:11: in com_google_api_gax_java_repositories
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/com_google_api_gax_java/repositories.bzl:139:14: in _maybe
Repository rule http_archive defined at:
  /home/runner/.cache/bazel/_bazel_runner/1f815583ebc8da6875b9074f31ba9a5d/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
([22](https://github.com/googleapis/gapic-generator-java/actions/runs/3717106771/jobs/6304147588#step:8:23):52:13) INFO: Analyzed 0 targets (1 packages loaded, 0 targets configured).
(22:52:13) INFO: Found 0 test targets...
(22:52:14) INFO: Elapsed time: 8.673s, Critical Path: 0.02s
(22:52:14) INFO: 1 process: 1 internal.
(22:52:14) ERROR: No test targets were found, yet testing was requested
Error: Process completed with exit code 4.

@suztomo
test class name not to have gapic-generator-java
3addbcf 
81 tests pass and 35 fail locally
@suztomo
Copy link
Member Author

suztomo commented Dec 17, 2022 •
edited
Loading

Around 31 tests rely on path written in test code. Fixing them to have 'gapic-generator-java' would pass the tests.

suztomo@suztomo:~/gapic-generator-java$ git diff
diff --git a/gapic-generator-java/src/test/java/com/google/api/generator/engine/JavaCodeGeneratorTest.java b/gapic-generator-java/src/test/java/com/google/api/generator/engine/JavaCodeGeneratorTest.java
index 3bbc0808d..d5d0fbb55 100644
--- a/gapic-generator-java/src/test/java/com/google/api/generator/engine/JavaCodeGeneratorTest.java
+++ b/gapic-generator-java/src/test/java/com/google/api/generator/engine/JavaCodeGeneratorTest.java
@@ -65,7 +65,7 @@ import org.junit.Test;
 public class JavaCodeGeneratorTest {
 
   private static final String GOLDENFILES_DIRECTORY =
-      "src/test/java/com/google/api/generator/engine/goldens/";
+      "gapic-generator-java/src/test/java/com/google/api/generator/engine/goldens/";
   // Create shared VaporReferences.
   private static final VaporReference stubRef = createVaporReference("com.google.gax.grpc", "Stub");
   private static final VaporReference bookKindRef =
diff --git a/gapic-generator-java/src/test/java/com/google/api/generator/gapic/protoparser/ServiceYamlParserTest.java b/gapic-generator-java/src/test/java/com/google/api/generator/gapic/protoparser/ServiceYamlParserTest.java
index 8df9ddcf8..e25c06429 100644
--- a/gapic-generator-java/src/test/java/com/google/api/generator/gapic/protoparser/ServiceYamlParserTest.java
+++ b/gapic-generator-java/src/test/java/com/google/api/generator/gapic/protoparser/ServiceYamlParserTest.java
@@ -24,7 +24,7 @@ import org.junit.Test;
 
 public class ServiceYamlParserTest {
 
-  private static final String YAML_DIRECTORY = "src/test/resources/";
+  private static final String YAML_DIRECTORY = "gapic-generator-java/src/test/resources/";
 
   @Test
   public void parseServiceYaml_basic() {

The failing test passed.

suztomo@suztomo:~/gapic-generator-java$ bazel test //:unit_com_google_api_generator_engine_JavaCodeGeneratorTest
(04:35:01) INFO: Current date is 2022-12-17
(04:35:04) DEBUG: /usr/local/google/home/suztomo/.cache/bazel/_bazel_suztomo/cd2df8cddf7b0bb247e02389468f06ab/external/io_grpc_grpc_java/java_grpc_library.bzl:76:14: in srcs attribute of @io_grpc_grpc_java//grpclb:load_balancer_java_grpc: Proto source with label @io_grpc_grpc_proto//:grpclb_load_balancer_proto should be in same package as consuming rule
(04:35:04) INFO: Analyzed target //:unit_com_google_api_generator_engine_JavaCodeGeneratorTest (184 packages loaded, 3378 targets configured).
(04:35:04) INFO: Found 1 test target...
Target //:unit_com_google_api_generator_engine_JavaCodeGeneratorTest up-to-date:
  bazel-bin/unit_com_google_api_generator_engine_JavaCodeGeneratorTest.jar
  bazel-bin/unit_com_google_api_generator_engine_JavaCodeGeneratorTest
(04:35:12) INFO: Elapsed time: 11.677s, Critical Path: 7.72s
(04:35:12) INFO: 3 processes: 1 internal, 1 linux-sandbox, 1 worker.
(04:35:12) INFO: Build completed successfully, 3 total actions
//:unit_com_google_api_generator_engine_JavaCodeGeneratorTest            PASSED in 0.9s

Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see whic(04:35:12) INFO: Build completed successfully, 3 total actions

However, such changes would make Maven test fail.

I need to learn constraints further.

  • Why Bazel related files need to come at root?
  • I can tweak the failing 31 tests in this manner for both Maven and Bazel. But is passing the Bazel test & Maven test really enough? Is there anything that would stop working if we delete src directory at the root?

@suztomo
Copy link
Member Author

suztomo commented Dec 19, 2022 •
edited
Loading

"ci / build (8)" and "ci / build (11)" succeeded after removing Bazel's unit tests, which Blake plans to remove in https://github.com/googleapis/gapic-generator-java/pull/918/files anyway.

Now coverage job, which relies on the unit tests, is remaining. As per Blake, there's a plan to move it to Maven-based.

@suztomo
ci: removing coverage job that was relying on Bazel unit tests
b1b62ef 
Bazel unit tests are being removed as part of Bazel to Maven
migration.
@suztomo
Copy link
Member Author

suztomo commented Dec 19, 2022

Retrying ci/build (8).

Run echo "Building Storage lib from generated source..."
  echo "Building Storage lib from generated source..."
  mkdir /tmp/java-storage
  bazel --batch build @com_google_googleapis//google/storage/v2:google-cloud-storage-v2-java
  tar zxvf bazel-bin/external/com_google_googleapis/google/storage/v2/google-cloud-storage-v2-java.tar.gz -C /tmp/java-storage
  pushd /tmp/java-storage/google-cloud-storage-v2-java
  ./gradlew clean build publishToMavenLocal sourcesJar allJars
  popd
  shell: /usr/bin/bash -e {0}
  env:
    JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/8.0.352-8/x64
    JAVA_HOME_8_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/8.0.352-8/x64
Building Storage lib from generated source...
([1](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:1)6:55:01) INFO: Current date is [2](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:2)022-12-19
(16:55:01) Loading: 
(16:55:01) Loading: 0 packages loaded
(16:55:01) Analyzing: target @com_google_googleapis//google/storage/v2:google-cloud-storage-v2-java (1 packages loaded, 0 targets configured)
(16:55:01) ERROR: /home/runner/work/gapic-generator-java/gapic-generator-java/rules_java_gapic/BUILD.bazel:7:8: //rules_java_gapic:gradlew: no such attribute 'extension' in '_real_pkg_tar' rule
(16:55:01) ERROR: /home/runner/work/gapic-generator-java/gapic-generator-java/rules_java_gapic/BUILD.bazel:7:8: //rules_java_gapic:gradlew: no such attribute 'files' in '_real_pkg_tar' rule
(16:55:01) ERROR: /home/runner/work/gapic-generator-java/gapic-generator-java/rules_java_gapic/BUILD.bazel:7:8: //rules_java_gapic:gradlew: no such attribute 'mode' in '_real_pkg_tar' rule
(16:55:01) ERROR: /home/runner/work/gapic-generator-java/gapic-generator-java/rules_java_gapic/BUILD.bazel:7:8: //rules_java_gapic:gradlew: no such attribute 'modes' in '_real_pkg_tar' rule
(16:55:01) ERROR: /home/runner/.cache/bazel/_bazel_runner/1f81558[3](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:3)ebc8da6875b907[4](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:4)f31ba9a[5](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:5)d/external/com_google_googleapis/google/storage/v2/BUILD.bazel:93:31: Target '//rules_java_gapic:gradlew' contains an error and its package is in error and referenced by '@com_google_googleapis//google/storage/v2:google-cloud-storage-v2-java'
(1[6](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:6):55:01) ERROR: Analysis of target '@com_google_googleapis//google/storage/v2:google-cloud-storage-v2-java' failed; build aborted: Analysis failed
(16:55:01) INFO: Elapsed time: 1.[9](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:9)29s
([16](https://github.com/googleapis/gapic-generator-java/actions/runs/3733310462/jobs/6334143457#step:9:17):55:01) INFO: 0 processes.
(16:55:01) FAILED: Build did NOT complete successfully (5 packages loaded, 5 targets configured)

@suztomo
Copy link
Member Author

suztomo commented Dec 19, 2022 •
edited
Loading

783b403 was passing for ci / build (8) but now it's failing.

It succeeds in my local:

suztomo@suztomo:~/gapic-generator-java$ bazel --batch build @com_google_googleapis//google/storage/v2:google-cloud-storage-v2-java
...
Target @com_google_googleapis//google/storage/v2:google-cloud-storage-v2-java up-to-date:
  bazel-bin/external/com_google_googleapis/google/storage/v2/google-cloud-storage-v2-java.tar.gz
(18:00:15) INFO: Elapsed time: 42.611s, Critical Path: 29.82s
(18:00:15) INFO: 222 processes: 35 internal, 153 linux-sandbox, 34 worker.
(18:00:15) INFO: Build completed successfully, 222 total actions

@emmileaf
Copy link
Contributor

I am seeing the same error in a different PR so the root cause is likely not from any commits here, but perhaps a github actions env or workflow change?

@suztomo
Copy link
Member Author

suztomo commented Dec 19, 2022

@emmileaf Thank you for info. Testing #1163

@suztomo suztomo mentioned this pull request Dec 19, 2022
Merged
@lqiu96 lqiu96 mentioned this pull request Dec 19, 2022
Merged
@suztomo
Copy link
Member Author

suztomo commented Dec 19, 2022

@emmileaf It's Bazel 6.0.0 #1163

@emmileaf
Copy link
Contributor

Thank you!

@suztomo suztomo added the automerge Merge the pull request once unit tests and other checks pass. label Dec 19, 2022
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@gcf-merge-on-green gcf-merge-on-green bot merged commit 473ee0b into main Dec 19, 2022
@gcf-merge-on-green gcf-merge-on-green bot deleted the remove_root_src branch December 19, 2022 23:42
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Dec 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@meltsufin meltsufin meltsufin approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@suztomo @emmileaf @meltsufin