Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: selectively generate libraries #3290

Merged
merged 8 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 39 additions & 7 deletions library_generation/cli/entry_point.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# limitations under the License.
import os
import sys

from typing import Optional
import click as click
from library_generation.generate_pr_description import generate_pr_descriptions
from library_generation.generate_repo import generate_from_yaml
Expand Down Expand Up @@ -51,6 +51,15 @@ def main(ctx):
metadata about library generation.
""",
)
@click.option(
"--includes",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

includes is a little ambiguous to me that the user may not know what is the info we are including. For example, is this api_shortname or library_name or something else? Just library_names or target_library_names(we are already using it in scripts) sounds more reasonable to me.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to library_names.

type=str,
default=None,
show_default=True,
help="""
A list of library names that will be generated, separated by comma.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add more docs here? I think it is basically library_name != null ? library_name : api_shortname from generation_config.yaml?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the comment.

""",
)
@click.option(
"--repository-path",
type=str,
Expand All @@ -77,33 +86,41 @@ def main(ctx):
def generate(
baseline_generation_config_path: str,
current_generation_config_path: str,
includes: Optional[str],
repository_path: str,
api_definitions_path: str,
):
"""
Compare baseline generation config and current generation config and
generate changed libraries based on current generation config with commit
history.
If `includes` is specified, libraries whose name can be found in the current
generation config will *also* be generated.

If baseline generation config is not specified but current generation
config is specified, generate all libraries based on current generation
config without commit history.
config is specified, generate all libraries if `includes` is not specified,
based on current generation config without commit history.
If `includes` is specified, only libraries whose name can be found in the
current generation config will be generated.

If current generation config is not specified but baseline generation
config is specified, raise FileNotFoundError because current generation
config should be the source of truth of library generation.

If both baseline generation config and current generation config are not
specified, generate all libraries based on the default generation config,
which is generation_config.yaml in the current working directory. Raise
FileNotFoundError if the default config does not exist.
which is generation_config.yaml in the current working directory.
If `includes` is specified, only libraries whose name can be found in the
default generation config will be generated.
Raise FileNotFoundError if the default config does not exist.

The commit history, if generated, will be available in
repository_path/pr_description.txt.
"""
__generate_repo_and_pr_description_impl(
baseline_generation_config_path=baseline_generation_config_path,
current_generation_config_path=current_generation_config_path,
includes=includes,
repository_path=repository_path,
api_definitions_path=api_definitions_path,
)
Expand All @@ -112,6 +129,7 @@ def generate(
def __generate_repo_and_pr_description_impl(
baseline_generation_config_path: str,
current_generation_config_path: str,
includes: Optional[str],
repository_path: str,
api_definitions_path: str,
):
Expand Down Expand Up @@ -146,14 +164,17 @@ def __generate_repo_and_pr_description_impl(
current_generation_config_path = os.path.abspath(current_generation_config_path)
repository_path = os.path.abspath(repository_path)
api_definitions_path = os.path.abspath(api_definitions_path)
include_libraries = _parse_library_name_from(includes)

if not baseline_generation_config_path:
# Execute full generation based on current_generation_config if
# Execute selective generation based on current_generation_config if
# baseline_generation_config is not specified.
# Do not generate pull request description.
generate_from_yaml(
config=from_yaml(current_generation_config_path),
repository_path=repository_path,
api_definitions_path=api_definitions_path,
target_library_names=include_libraries,
)
return

Expand All @@ -164,12 +185,17 @@ def __generate_repo_and_pr_description_impl(
baseline_config=from_yaml(baseline_generation_config_path),
current_config=from_yaml(current_generation_config_path),
)
# pass None if we want to fully generate the repository.
# Pass None if we want to fully generate the repository.
target_library_names = (
config_change.get_changed_libraries()
if not _needs_full_repo_generation(config_change=config_change)
else None
)
# Generate the union of two library lists if neither of the library list
# indicates a full generation.
# The deduplication is done in `generate_from_yaml`.
if (target_library_names is not None) and (include_libraries is not None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this making the logic more complicated, if include_libraries is provided, it should just override target_library_names. Especially we plan to remove target_library_names in the future.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I also found it was a bit of confusing in this scenario.

I changed this to library_names takes preference if specified.

target_library_names.extend(include_libraries)
generate_from_yaml(
config=config_change.current_config,
repository_path=repository_path,
Expand All @@ -191,6 +217,12 @@ def _needs_full_repo_generation(config_change: ConfigChange) -> bool:
return not current_config.is_monorepo() or current_config.contains_common_protos()


def _parse_library_name_from(includes: str) -> Optional[list[str]]:
if includes is None:
return None
return includes.split(",")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe trim the library name?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.



@main.command()
@click.option(
"--generation-config-path",
Expand Down
5 changes: 2 additions & 3 deletions library_generation/generate_repo.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import shutil

from typing import Optional
import library_generation.utils.utilities as util
from library_generation.generate_composed_library import generate_composed_library
from library_generation.model.generation_config import GenerationConfig
Expand All @@ -26,7 +25,7 @@ def generate_from_yaml(
config: GenerationConfig,
repository_path: str,
api_definitions_path: str,
target_library_names: list[str] = None,
target_library_names: Optional[list[str]],
) -> None:
"""
Based on the generation config, generates libraries via
Expand Down
Loading
Loading