Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: script to create a new module #8517

Merged
merged 15 commits into from
Oct 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 32 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,34 @@
# Packages
# Python Packages
# Exerpt from GitHub's recommendation
# https://github.com/github/gitignore/blob/main/Python.gitignore
.Python
dist
bin
var
sdist
target
eggs/
.eggs/
sdist/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Python utilities
__pycache__/
*.py[cod]
*$py.class

# Python environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/


# Unit test / coverage reports
.coverage
Expand All @@ -13,6 +38,7 @@ nosetests.xml
# Translations
*.mo

# Irrelevant files generated by IDEs and OSes
.metadata
.project
.pydevproject
Expand All @@ -28,7 +54,9 @@ docs/
# API key file containing value of GOOGLE_API_KEY for integration tests
api_key

# Python utilities
*.pyc
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pyc is covered by *.py[cod]

# New client generation
generation/new_client/workspace

# Monorepo repository generation
monorepo/

monorepo/
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This moved to the end.

174 changes: 174 additions & 0 deletions generation/new_client/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
# New Google Client Library Generation

The script allows you to create a new client library module in the
google-cloud-java monorepo.

**This tool is for repository maintainers only. Not for library users.**

## Prerequisites

This section is only needed for the first run of this script. If it's already
done, go to "Run client generation script" section.


### Environment

Use Linux environment.

Install Docker.

### Checkout google-cloud-java repository

```
$ git clone https://github.com/googleapis/google-cloud-java
```

### Install pyenv

Install pyenv

```
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer \
| bash
```

Follow the instruction in the output above to append lines in `$HOME/.bashrc`.
Logout the shell and login again.

Confirm pyenv installation succeeded:

```
suztomo@suztomo:~$ pyenv
pyenv 2.3.4
Usage: pyenv <command> [<args>]

Some useful pyenv commands are:
activate Activate virtual environment
commands List all available pyenv commands
deactivate Deactivate virtual environment
...
```

### Install Python 3.9 via pyenv

```
suztomo@suztomo:~$ pyenv install 3.9.13
Downloading Python-3.9.13.tar.xz...
-> https://www.python.org/ftp/python/3.9.13/Python-3.9.13.tar.xz
Installing Python-3.9.13...
WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib?
Installed Python-3.9.13 to /usr/local/google/home/suztomo/.pyenv/versions/3.9.13
```

Conform `python3.9` command is available:

```
$ pyenv local 3.9.13
$ which python3.9
/usr/local/google/home/suztomo/.pyenv/shims/python3.9
```

### Install Python packages

At the root of google-cloud-java repository clone, run:

```
$ python3.9 -m pip install -r generation/new_client/requirements.txt
```

## Run client generation script

You will run new-client.py script with the following parameters.
Collect them from the ticket before running the command.

### API short name

For convenience of the subsequent commands, define a variable for API short name.
Get the value from the DevRel Services page (Example: `apikeys`):

### proto path

The script takes "proto path" parameter. This is path from google3's root to the
directory that contains versions (e.g., "v1" or "v2"). For example, if we
have

> Link to protos: http://google3/google/api/apikeys/v2/apikeys.proto

then the "proto path" value we supply to the command is "google/api/apikeys".

### Name pretty

Pick name from the display name in the DevRel Services site.
For example, "API Keys API" in
https://devrel.corp.google.com/admin/products/479.

### Product Docs

Find product document URL in the DevRel Services site.
For example, "https://cloud.google.com/api-keys/" in
https://devrel.corp.google.com/admin/products/479.

### API description

Find the short description of the API. Usually the first sentence in the product
document above.

### Release level

The first client generation is always "preview".

### Transport

Transport layer. Use "grpc".


### Language

Use "java"

### Monorepo URL

Use "https://github.com/googleapis/google-cloud-java"


### Example arguments

Run `new-client.py` with the arguments above:

```
$ cd generation/new_client
$ python3.9 new-client.py generate \
--api_shortname=apikeys \
--proto-path=google/api/apikeys \
--name-pretty="API Keys API" \
--product-docs="https://cloud.google.com/api-keys/" \
--api-description="API Keys lets you create and manage your API keys for your projects." \
--release-level=preview \
--transport=grpc \
--language=java \
--monorepo-url="https://github.com/googleapis/google-cloud-java"
```

The command creates `workspace` directory in which it prepares the changes for
the new module in the monorepo. At the end (~ 10 minutes), it tells you to
create a pull request in the monorepo:

```
...
Prepared new library in workspace/monorepo/java-apikeys
Please create a pull request from that directory:
$ cd /usr/local/google/home/suztomo/google-cloud-java/generation/new_client/workspace/monorepo
$ gh pr create --title 'feat: [apikeys] new module for apikeys'
```

Create a pull request from the change.

# Principles

The script should finish creating a pull request even when the newly created
module fails to compile. This gives the user flexibility to fix things in the
created pull request.




Loading