Skip to content

Commit

Permalink
feat: add entry point (#2616)
Browse files Browse the repository at this point in the history
In this PR:
- Create `entry_point.py` to combine `generate_repo.py` and
`generate_pr_description.py`.
- Change Integration test.

Follow up of #2604.

For the goal of series of PRs, please refer to [improvement
proposal](https://docs.google.com/document/d/1JiCcG3X7lnxaJErKe0ES_JkyU7ECb40nf2Xez3gWvuo/edit?tab=t.g3vua2kd06gx#bookmark=id.72s3ukwwzevo).
  • Loading branch information
JoeWang1127 authored Apr 5, 2024
1 parent 56775a6 commit b19fa33
Show file tree
Hide file tree
Showing 14 changed files with 536 additions and 244 deletions.
53 changes: 30 additions & 23 deletions library_generation/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Generate a repository containing GAPIC Client Libraries

The script, `generate_repo.py`, allows you to generate a repository containing
GAPIC client libraries (a monorepo, for example, google-cloud-java) from a
configuration file.
The script, `entry_point.py`, allows you to generate a repository containing
GAPIC client libraries with googleapis commit history (a monorepo, for example,
google-cloud-java) from a configuration file.

## Environment

Expand All @@ -17,30 +17,27 @@ In order to generate a version for each library, a versions.txt has to exist
in `repository_path`.
Please refer to [Repository path](#repository-path--repositorypath---optional) for more information.

## Parameters to generate a repository using `generate_repo.py`
## Parameters to generate a repository using `entry_point.py`

### Generation configuration yaml (`generation_config_yaml`)
### Baseline generation configuration yaml (`baseline_generation_config`)

A path to a configuration file containing parameters to generate the repository.
Please refer [Configuration to generate a repository](#configuration-to-generate-a-repository)
for more information.

### Target library API shortname (`target_library_api_shortname`), optional
An absolute or relative path to a generation_config.yaml.
This config file is used for commit history generation, not library
generation.

If specified, the libray whose `api_shortname` equals to `target_library_api_shortname`
will be generated; otherwise all libraries in the configuration file will be
generated.
This can be useful when you just want to generate one library for debugging
purposes.
### Current generation configuration yaml (`current_generation_config`)

The default value is an empty string, which means all libraries will be generated.
An absolute or relative path to a configuration file containing parameters to
generate the repository.
Please refer [Configuration to generate a repository](#configuration-to-generate-a-repository)
for more information.

### Repository path (`repository_path`), optional

The path to where the generated repository goes.

The default value is the current working directory when running the script.
For example, `cd google-cloud-java && python generate_repo.py ...` without
For example, `cd google-cloud-java && python entry_point.py ...` without
specifying the `--repository_path` option will modify the `google-cloud-java`
repository the user `cd`'d into.

Expand All @@ -49,7 +46,9 @@ right version for each library.
Please refer [here](go/java-client-releasing#versionstxt-manifest) for more info
of versions.txt.

## Output of `generate_repo.py`
## Output of `entry_point.py`

### GAPIC libraries

For each module (e.g. `google-cloud-java/java-asset`), the following files/folders
will be created/modified:
Expand All @@ -73,6 +72,11 @@ will be created/modified:
| pom.xml (repo root dir) | Always generated from inputs |
| versions.txt | New entries will be added if they don’t exist |

### googleapis commit history

If both `baseline_generation_config` and `current_generation_config` are
specified, and they contain different googleapis commit, the commit history will
be generated into `pr_description.txt` in the `repository_path`.

## Configuration to generate a repository

Expand Down Expand Up @@ -184,20 +188,23 @@ libraries:
- proto_path: google/cloud/asset/v1p7beta1
```
## An example to generate a repository using `generate_repo.py`
## An example to generate a repository using `entry_point.py`

```bash
# install python module (allows the `library_generation` module to be imported from anywhere)
python -m pip install -r library_generation/requirements.in
# install library_generation module
python -m pip install library_generation
# generate the repository
python -m library_generation/generate_repo.py generate \
--generation-config-yaml=/path/to/config-file \
python -m library_generation/entry_point.py generate \
--baseline-generation-config=/path/to/baseline_config_file \
--current-generation-config=/path/to/current_config_file \
--repository-path=/path/to/repository
```

## An example of generated repository using `generate_repo.py`
## An example of generated repository using `entry_point.py`

If you run `generate_repo.py` with the example [configuration](#an-example-of-generation-configuration)
If you run `entry_point.py` with the example [configuration](#an-example-of-generation-configuration)
shown above, the repository structure is:
```
$repository_path
Expand Down
141 changes: 141 additions & 0 deletions library_generation/cli/entry_point.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os

import click as click
from library_generation.generate_pr_description import generate_pr_descriptions
from library_generation.generate_repo import generate_from_yaml
from library_generation.model.generation_config import from_yaml
from library_generation.utils.generation_config_comparator import compare_config


@click.group(invoke_without_command=False)
@click.pass_context
@click.version_option(message="%(version)s")
def main(ctx):
pass


@main.command()
@click.option(
"--baseline-generation-config",
required=False,
default=None,
type=str,
help="""
Absolute or relative path to a generation_config.yaml.
This config file is used for commit history generation, not library
generation.
""",
)
@click.option(
"--current-generation-config",
required=False,
default=None,
type=str,
help="""
Absolute or relative path to a generation_config.yaml that contains the
metadata about library generation.
""",
)
@click.option(
"--repository-path",
type=str,
default=".",
show_default=True,
help="""
The repository path to which the generated files
will be sent.
If not specified, the repository will be generated to the current working
directory.
""",
)
def generate(
baseline_generation_config: str,
current_generation_config: str,
repository_path: str,
):
"""
Compare baseline generation config and current generation config and
generate changed libraries based on current generation config with commit
history.
If baseline generation config is not specified but current generation
config is specified, generate all libraries based on current generation
config without commit history.
If current generation config is not specified but baseline generation
config is specified, raise FileNotFoundError because current generation
config should be the source of truth of library generation.
If both baseline generation config and current generation config are not
specified, generate all libraries based on the default generation config,
which is generation_config.yaml in the current working directory. Raise
FileNotFoundError if the default config does not exist.
The commit history, if generated, will be available in
repository_path/pr_description.txt.
"""
default_generation_config = f"{os.getcwd()}/generation_config.yaml"

if baseline_generation_config is None and current_generation_config is None:
if not os.path.isfile(default_generation_config):
raise FileNotFoundError(
f"{default_generation_config} does not exist. "
"A valid generation config has to be passed in as "
"current_generation_config or exist in the current working "
"directory."
)
current_generation_config = default_generation_config
elif current_generation_config is None:
raise FileNotFoundError(
"current_generation_config is not specified when "
"baseline_generation_config is specified. "
"current_generation_config should be the source of truth of "
"library generation."
)

current_generation_config = os.path.abspath(current_generation_config)
repository_path = os.path.abspath(repository_path)
if not baseline_generation_config:
# Execute full generation based on current_generation_config if
# baseline_generation_config is not specified.
# Do not generate pull request description.
generate_from_yaml(
config=from_yaml(current_generation_config),
repository_path=repository_path,
)
return

# Compare two generation configs and only generate changed libraries.
# Generate pull request description.
baseline_generation_config = os.path.abspath(baseline_generation_config)
config_change = compare_config(
baseline_config=from_yaml(baseline_generation_config),
current_config=from_yaml(current_generation_config),
)
generate_from_yaml(
config=config_change.current_config,
repository_path=repository_path,
target_library_names=config_change.get_changed_libraries(),
)
generate_pr_descriptions(
config=config_change.current_config,
baseline_commit=config_change.baseline_config.googleapis_commitish,
description_path=repository_path,
)


if __name__ == "__main__":
main()
Loading

0 comments on commit b19fa33

Please sign in to comment.