feat: generate proto-only repository #2720

JoeWang1127 · 2024-05-02T17:18:32Z

In this PR:

Enable hermetic build script to generate java-common-protos and java-iam in this repo.
Change libraries_bom_version to optional.
Refactor unit tests.
Remove java-common-protos/codecov.yaml and java-iam/codecov.yaml as they are no longer needed.

blakeli0 · 2024-05-09T21:18:08Z

java-iam/README.md is excluded from templating, should we not generate it?

I don't think it makes sense for both java-iam and java-common-protos to have READMEs, I think we can remove them and exclude them from generation. WDYT @suztomo ?

JoeWang1127 · 2024-05-14T14:56:03Z

library_generation/generate_repo.py

        )

-    # we skip monorepo_postprocessing if not in a monorepo
-    if not config.is_monorepo():
+    if not config.is_monorepo() or config.contains_common_protos():


Only perform monorepo post processing if this is a gapic monorepo.

diegomarquezp

LGTM, just some minor comments.

diegomarquezp · 2024-05-15T14:53:53Z

library_generation/generate_repo.py

@@ -18,6 +18,8 @@
 from library_generation.model.library_config import LibraryConfig
 from library_generation.utils.monorepo_postprocessor import monorepo_postprocessing

+PROTO_ONLY_LIBRARIES = ["common-protos"]


I believe this one should also be changed to "common proto libraries"

This variable is actually not used, I removed it.

diegomarquezp · 2024-05-15T14:56:56Z

library_generation/model/generation_config.py

@@ -61,6 +63,15 @@ def get_proto_path_to_library_name(self) -> dict[str, str]:
    def is_monorepo(self) -> bool:
        return len(self.libraries) > 1

+    def contains_common_protos(self) -> bool:


We have one method to set the value and one to compute, plus we store a variable in the config object. Maybe we can put the logic of the setter in the getter method and compute the value once in the field we create in the constructor. That would leave us with one less method.

blakeli0 · 2024-05-16T03:45:09Z

library_generation/model/generation_config.py

@@ -20,6 +20,7 @@
 REPO_LEVEL_PARAMETER = "Repo level parameter"
 LIBRARY_LEVEL_PARAMETER = "Library level parameter"
 GAPIC_LEVEL_PARAMETER = "GAPIC level parameter"
+COMMON_PROTOS = "common-protos"


Suggested change

COMMON_PROTOS = "common-protos"

COMMON_PROTOS_LIBRARY_NAME = "common-protos"

blakeli0 · 2024-05-16T04:15:05Z

library_generation/generate_repo.py

        generate_composed_library(
            config_path=config_path,
            config=config,
            library_path=library_path,
            library=library,
            output_folder=repo_config.output_folder,
            versions_file=repo_config.versions_file,
+            gapic_repo=not config.contains_common_protos(),


Why do we call this gapic_repo? I see that config is already being passed around, can we just use config.contains_common_protos()? I feel it is easier to understand.

I removed this parameter as we already pass the config.

blakeli0 · 2024-05-16T04:24:01Z

library_generation/test/model/gapic_inputs_unit_tests.py

Thanks for backfilling this test!

blakeli0 · 2024-05-16T20:45:04Z

library_generation/utils/utilities.py

@@ -250,6 +252,8 @@ def generate_prerequisite_files(
        "library_type": library.library_type,
        "requires_billing": library.requires_billing,
    }
+    if config.contains_common_protos():


nit: Can we add a comment saying that this is for java-common-protos and java-iam modules in sdk-platform-java? And/or change the logic to if repo == "googleapis/sdk-platform-java"?

blakeli0 · 2024-05-16T20:46:13Z

library_generation/utils/utilities.py

-        if config.is_monorepo()
-        else f"googleapis/{language}-{library_name}"
-    )
+    if config.contains_common_protos():


Future consideration: Maybe we want to introduce a repo level config called repo or repo_name to represent this.

blakeli0

LGTM.

sonarcloud · 2024-05-16T21:36:11Z

Quality Gate passed for 'gapic-generator-java-root'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

sonarcloud · 2024-05-16T21:39:15Z

Quality Gate passed for 'java_showcase_integration_tests'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

In this PR: - Enable hermetic build script to generate `java-common-protos` and `java-iam` in this repo. - Change `libraries_bom_version` to optional. - Refactor unit tests. - Remove `java-common-protos/codecov.yaml` and `java-iam/codecov.yaml` as they are no longer needed.

🤖 I have created a release *beep* *boop* --- <details><summary>2.41.0</summary> ## [2.41.0](v2.40.1...v2.41.0) (2024-05-31) ### Features * add a github client ([#2747](#2747)) ([f8ea0a0](f8ea0a0)) * generate proto-only repository ([#2720](#2720)) ([f7a5161](f7a5161)) ### Bug Fixes * [#2785](#2785). ([599f5da](599f5da)) ### Dependencies * update dependency com.google.api-client:google-api-client-bom to v2.6.0 ([#2782](#2782)) ([5bc8928](5bc8928)) * update dependency com.google.cloud.opentelemetry:detector-resources-support to v0.29.0 ([#2831](#2831)) ([6c1dbfc](6c1dbfc)) * update dependency com.google.code.gson:gson to v2.11.0 ([#2786](#2786)) ([91f3254](91f3254)) * update dependency com.google.code.gson:gson to v2.11.0 ([#2787](#2787)) ([e81893c](e81893c)) * update dependency com.google.errorprone:error_prone_annotations to v2.28.0 ([#2835](#2835)) ([b8f11b1](b8f11b1)) * update dependency com.google.errorprone:error_prone_annotations to v2.28.0 ([#2838](#2838)) ([5c46f3f](5c46f3f)) * update dependency net.bytebuddy:byte-buddy to v1.14.16 ([#2797](#2797)) ([dfedafc](dfedafc)) * update dependency net.bytebuddy:byte-buddy to v1.14.17 ([#2828](#2828)) ([6eb9041](6eb9041)) * update dependency org.checkerframework:checker-qual to v3.43.0 ([#2730](#2730)) ([b7fa736](b7fa736)) * update dependency requests to v2.32.0 [security] ([#2791](#2791)) ([c2ea6cc](c2ea6cc)) * update dependency watchdog to v4.0.1 ([#2800](#2800)) ([d5771dd](d5771dd)) * update google api dependencies ([#2672](#2672)) ([6643536](6643536)) * update google http client dependencies to v1.44.2 ([#2783](#2783)) ([dee7e00](dee7e00)) * update googleapis/java-cloud-bom digest to 59c776b ([#2827](#2827)) ([03b3eb4](03b3eb4)) * update netty dependencies to v4.1.110.final ([#2796](#2796)) ([d1aaa68](d1aaa68)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

chore: add generation config

53e0a2b

product-auto-label bot added the size: s Pull request size is small. label May 2, 2024

JoeWang1127 added 6 commits May 2, 2024 13:21

Merge branch 'main' into chore/add-generation-config

7541b67

add gapics

4f5118c

Merge branch 'main' into chore/add-generation-config

6629f2f

add java-iam

b8be0b2

fix proto_path

7f48e08

rename OwlBot.yaml

0026eed

product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels May 2, 2024

JoeWang1127 added 15 commits May 2, 2024 23:23

change command

78dd8d7

Merge branch 'main' into chore/add-generation-config

788e30c

restore change

971867e

change source dir

f3a6fbd

change source

ddce346

change to snapshot

8c21004

change readme template

7da4e21

change copyright year

abcc81b

do not apply repo level postprocessing if it has proto-only libraries

9a65467

generate proto-only repo

e19c0ff

do not generate api-id

edc09e2

Merge branch 'main' into chore/add-generation-config

4321f11

refactor unit tests

a4f945f

add unit test to verify proto-only BUILD

8b3563b

change readme template

396ea8e

JoeWang1127 changed the title ~~chore: add generation config~~ feat: generate proto-only repository May 8, 2024

JoeWang1127 added 2 commits May 8, 2024 14:37

fix unit test

905ce01

Merge branch 'main' into chore/add-generation-config

ed8e811

merge main

afa2f2f

JoeWang1127 commented May 14, 2024

View reviewed changes

Merge branch 'main' into chore/add-generation-config

bd7859f

JoeWang1127 requested a review from diegomarquezp May 14, 2024 17:47

JoeWang1127 added 4 commits May 14, 2024 19:31

Merge branch 'main' into chore/add-generation-config

c18090f

Merge branch 'main' into chore/add-generation-config

b46723d

Merge branch 'main' into chore/add-generation-config

685905a

Merge branch 'main' into chore/add-generation-config

7748828

diegomarquezp approved these changes May 15, 2024

View reviewed changes

JoeWang1127 added 4 commits May 15, 2024 16:53

remove variable

a25e604

Merge branch 'main' into chore/add-generation-config

958ef5a

remove setter

efa3efa

add break

c76524d

blakeli0 reviewed May 16, 2024

View reviewed changes

JoeWang1127 added 2 commits May 16, 2024 12:12

Merge branch 'main' into chore/add-generation-config

c4731d1

remove a parameter

7385de4

JoeWang1127 requested a review from blakeli0 May 16, 2024 12:55

blakeli0 reviewed May 16, 2024

View reviewed changes

blakeli0 approved these changes May 16, 2024

View reviewed changes

JoeWang1127 added 2 commits May 16, 2024 21:28

refactor

534ae9c

Merge branch 'main' into chore/add-generation-config

1751738

JoeWang1127 merged commit f7a5161 into main May 16, 2024
48 checks passed

JoeWang1127 deleted the chore/add-generation-config branch May 16, 2024 21:52

release-please bot mentioned this pull request May 16, 2024

chore(main): release 2.41.0 #2780

Closed

release-please bot mentioned this pull request May 24, 2024

chore(main): release 2.41.0 #2820

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: generate proto-only repository #2720

feat: generate proto-only repository #2720

JoeWang1127 commented May 2, 2024 •

edited

Loading

blakeli0 commented May 9, 2024

JoeWang1127 May 14, 2024

diegomarquezp left a comment

diegomarquezp May 15, 2024

JoeWang1127 May 15, 2024

diegomarquezp May 15, 2024

blakeli0 May 16, 2024

JoeWang1127 May 16, 2024

blakeli0 May 16, 2024

JoeWang1127 May 16, 2024

blakeli0 May 16, 2024

blakeli0 May 16, 2024

JoeWang1127 May 16, 2024

blakeli0 May 16, 2024

blakeli0 left a comment

sonarcloud bot commented May 16, 2024

sonarcloud bot commented May 16, 2024

	COMMON_PROTOS = "common-protos"
	COMMON_PROTOS_LIBRARY_NAME = "common-protos"

feat: generate proto-only repository #2720

feat: generate proto-only repository #2720

Conversation

JoeWang1127 commented May 2, 2024 • edited Loading

blakeli0 commented May 9, 2024

Choose a reason for hiding this comment

diegomarquezp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blakeli0 left a comment

Choose a reason for hiding this comment

sonarcloud bot commented May 16, 2024

Quality Gate passed for 'gapic-generator-java-root'

sonarcloud bot commented May 16, 2024

Quality Gate passed for 'java_showcase_integration_tests'

JoeWang1127 commented May 2, 2024 •

edited

Loading