Skip to content

Commit

Permalink
Generate Tekton Pipeline Python SDK
Browse files Browse the repository at this point in the history
  • Loading branch information
jinchihe authored and tekton-robot committed Oct 30, 2020
1 parent f220993 commit f9f49d9
Show file tree
Hide file tree
Showing 258 changed files with 27,052 additions and 0 deletions.
5 changes: 5 additions & 0 deletions sdk/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# The OWNERS file is used by prow to automatically merge approved PRs.

approvers:
- jinchihe

2 changes: 2 additions & 0 deletions sdk/hack/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
openapi-generator-cli.jar
swagger.json
55 changes: 55 additions & 0 deletions sdk/hack/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@

# Readme for Generating Tekton Pipeline SDK

The guide shows how to generate the openapi model and swagger.json file from Tekton Pipeline types using `openapi-gen` and generate Tekton Pipeline Python SDK Client for the Python object models using `openapi-generator`. Also show how to upload the Tekton Pipeline SDK to Pypi.

## Update openapi spec and swagger file.

Download `tektoncd/pipeline` repository, and execute the below script to generate openapi spec and swagger file.

```
./hack/update-openapigen.sh
```
After executing, the `openapi_generated.go` and `swagger.json` are refreshed under `pkg/apis/pipeline/v1beta1/`.

And then copy the `pkg/apis/pipeline/v1beta1/swagger.json` to the `sdk/hack` in this repo. If not copy, the `sdk-gen.sh` will download from github directly.

## Generate Tekton Pipeline Python SDK

Execute the script `/sdk/hack/sdk-gen.sh` to install openapi-generator and generate Tekton Pipeline Python SDK.

```
./sdk/hack/sdk-gen.sh
```
After the script execution, the Tekton Pipeline Python SDK is generated in the `./sdk/python` directory. Some files such as [README](../python/README.md) and setup.py need to be merged manually after the script execution.

## (Optional) Refresh Python SDK in the Pypi

Navigate to `sdk/python/tekton` directory.

1. Install `twine`:

```bash
pip install twine
```

2. Update the Tekton Pipeline Python SDK version in the [setup.py](../python/setup.py).

3. Create some distributions in the normal way:

```bash
python setup.py sdist bdist_wheel
```

4. Upload with twine to [Test PyPI](https://packaging.python.org/guides/using-testpypi/) and verify things look right. `Twine` will automatically prompt for your username and password:
```bash
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
username: ...
password:
...
```

5. Upload to [PyPI](https://pypi.org/project/tekton-pipeline/):
```bash
twine upload dist/*
```
14 changes: 14 additions & 0 deletions sdk/hack/boilerplate.python.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright 2020 The Tekton Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

78 changes: 78 additions & 0 deletions sdk/hack/sdk-gen.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#!/usr/bin/env bash

# Copyright 2020 The Tekton Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset

OPENAPI_GEN_URL="https://repo1.maven.org/maven2/org/openapitools/openapi-generator-cli/4.3.1/openapi-generator-cli-4.3.1.jar"
OPENAPI_GEN_JAR="sdk/hack/openapi-generator-cli.jar"
SWAGGER_CODEGEN_CONF="sdk/hack/swagger_config.json"
SWAGGER_CODEGEN_FILE="sdk/hack/swagger.json"
SWAGGER_CODEGEN_SOURCE="https://github.com/tektoncd/pipeline/tree/master/pkg/apis/pipeline/v1beta1/swagger.json"
SDK_OUTPUT_PATH="./sdk/python"

echo "Check the swagger.json file ..."
if [ ! -f ${SWAGGER_CODEGEN_FILE} ]
then
wget -O ${SWAGGER_CODEGEN_FILE} ${SWAGGER_CODEGEN_SOURCE}
fi

echo "Downloading the swagger-codegen JAR package ..."
if [ ! -f ${OPENAPI_GEN_JAR} ]
then
wget -O ${OPENAPI_GEN_JAR} ${OPENAPI_GEN_URL}
fi

echo "Generating Python SDK for Tekton Pipeline ..."
java -jar ${OPENAPI_GEN_JAR} generate -i ${SWAGGER_CODEGEN_FILE} -g python -o ${SDK_OUTPUT_PATH} -c ${SWAGGER_CODEGEN_CONF}

echo "Adding Python boilerplate message ..."
for i in $(find ./sdk/python -name *.py)
do
if ! grep -q Copyright $i
then
cat sdk/hack/boilerplate.python.txt $i >$i.new && mv $i.new $i
fi
done

echo "Replace Kubernetes document link ..."
MAPPING_LIST=`grep V1 ${SWAGGER_CODEGEN_CONF} |awk -F '"' '{print $2}'`
K8S_URL='https://github.com/kubernetes-client/python/blob/master/kubernetes/docs'
for map in ${MAPPING_LIST}
do
sed -i'.bak' -e "s@($map.md)@($K8S_URL/$map.md)@g" ./sdk/python/docs/*
rm -rf ./sdk/python/docs/*.bak
done

echo "Update some specify files ..."
git checkout ${SDK_OUTPUT_PATH}/setup.py
git checkout ${SDK_OUTPUT_PATH}/requirements.txt
# Better to merge README file munally.
#git checkout ${SDK_OUTPUT_PATH}/README.md

if ! grep -q "TektonClient" ${SDK_OUTPUT_PATH}/tekton/__init__.py
then
echo "from tekton.api.tekton_client import TektonClient" >> ${SDK_OUTPUT_PATH}/tekton/__init__.py
fi

if ! grep -q "constants" ${SDK_OUTPUT_PATH}/tekton/__init__.py
then
echo "from tekton.constants import constants" >> ${SDK_OUTPUT_PATH}/tekton/__init__.py
fi

echo "Tekton Pipeline Python SDK is generated successfully to folder ${SDK_OUTPUT_PATH}/."


34 changes: 34 additions & 0 deletions sdk/hack/swagger_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"packageName" : "tekton",
"projectName" : "tekton",
"packageVersion": "0.1",
"importMappings": {
"V1Container": "from kubernetes.client import V1Container",
"V1ObjectMeta": "from kubernetes.client import V1ObjectMeta",
"V1ListMeta": "from kubernetes.client import V1ListMeta",
"V1Volume": "from kubernetes.client import V1Volume",
"V1ResourceRequirements": "from kubernetes.client import V1ResourceRequirements",
"V1ContainerPort": "from kubernetes.client import V1ContainerPort",
"V1EnvFromSource": "from kubernetes.client import V1EnvFromSource",
"V1EnvVar": "from kubernetes.client import V1EnvVar",
"V1Lifecycle": "from kubernetes.client import V1Lifecycle",
"V1Probe": "from kubernetes.client import V1Probe",
"V1SecurityContext": "from kubernetes.client import V1SecurityContext",
"V1VolumeDevice": "from kubernetes.client import V1VolumeDevice",
"V1VolumeMount": "from kubernetes.client import V1VolumeMount",
"V1ConfigMapVolumeSource": "from kubernetes.client import V1ConfigMapVolumeSource",
"V1EmptyDirVolumeSource": "from kubernetes.client import V1EmptyDirVolumeSource",
"V1PersistentVolumeClaim": "from kubernetes.client import V1PersistentVolumeClaim",
"V1PersistentVolumeClaimVolumeSource": "from kubernetes.client import V1PersistentVolumeClaimVolumeSource",
"V1SecretVolumeSource": "from kubernetes.client import V1SecretVolumeSource",
"V1ContainerStateRunning": "from kubernetes.client import V1ContainerStateRunning",
"V1ContainerStateTerminated": "from kubernetes.client import V1ContainerStateTerminated",
"V1ContainerStateWaiting": "from kubernetes.client import V1ContainerStateWaiting",
"V1ContainerState": "from kubernetes.client import V1ContainerState",
"V1Affinity": "from kubernetes.client import V1Affinity",
"V1LocalObjectReference": "from kubernetes.client import V1LocalObjectReference",
"V1PodDNSConfig": "from kubernetes.client import V1PodDNSConfig",
"V1PodSecurityContext": "from kubernetes.client import V1PodSecurityContext",
"V1Toleration": "from kubernetes.client import V1Toleration"
}
}
75 changes: 75 additions & 0 deletions sdk/python/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
venv/
.python-version

# Translations
*.mo
*.pot

# Django stuff:
*.log

# Sphinx documentation
docs/_build/

# PyBuilder
target/

#Ipython Notebook
.ipynb_checkpoints

#SDK
.swagger-codegen-ignore
.swagger-codegen/
.travis.yml
git_push.sh
tox.ini
.gitlab-ci.yml

.openapi-generator/
git_push.sh
23 changes: 23 additions & 0 deletions sdk/python/.openapi-generator-ignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Swagger Codegen Ignore
# Generated by swagger-codegen https://github.com/swagger-api/swagger-codegen

# Use this file to prevent files from being overwritten by the generator.
# The patterns follow closely to .gitignore or .dockerignore.

# As an example, the C# client generator defines ApiClient.cs.
# You can make changes and tell Swagger Codgen to ignore just this file by uncommenting the following line:
#ApiClient.cs

# You can match any string of characters against a directory, file or extension with a single asterisk (*):
#foo/*/qux
# The above matches foo/bar/qux and foo/baz/qux, but not foo/bar/baz/qux

# You can recursively match patterns against a directory, file or extension with a double asterisk (**):
#foo/**/qux
# This matches foo/bar/qux, foo/baz/qux, and foo/bar/baz/qux

# You can also negate patterns with an exclamation (!).
# For example, you can ignore all files in a docs folder with the file extension .md:
#docs/*.md
# Then explicitly reverse the ignore rule for a single file:
#!docs/README.md
Loading

0 comments on commit f9f49d9

Please sign in to comment.