Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate sdk for v2 #434

Merged
merged 5 commits into from
Nov 30, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 5 additions & 13 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,21 @@ module github.com/kubeflow/mpi-operator
go 1.13

require (
github.com/emicklei/go-restful v2.9.6+incompatible // indirect
github.com/go-logr/logr v0.3.0 // indirect
github.com/go-openapi/spec v0.20.3
github.com/google/go-cmp v0.5.6 // indirect
github.com/googleapis/gnostic v0.5.1 // indirect
github.com/hashicorp/golang-lru v0.5.4 // indirect
github.com/imdario/mergo v0.3.10 // indirect
github.com/kubeflow/common v0.4.0
github.com/onsi/ginkgo v1.14.1 // indirect
github.com/onsi/gomega v1.10.2 // indirect
github.com/kubeflow/mpi-operator/v2 v2.0.0-00010101000000-000000000000
github.com/prometheus/client_golang v1.10.0
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e // indirect
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e // indirect
google.golang.org/appengine v1.6.6 // indirect
k8s.io/api v0.19.9
k8s.io/apimachinery v0.19.9
k8s.io/apiserver v0.19.9
k8s.io/client-go v0.19.9
k8s.io/klog v1.0.0
k8s.io/kube-openapi v0.0.0-20200805222855-6aeccd4b50c6
k8s.io/sample-controller v0.19.9
k8s.io/utils v0.0.0-20200912215256-4140de9c8800 // indirect
volcano.sh/apis v1.2.0-k8s1.19.6
)

replace k8s.io/code-generator => k8s.io/code-generator v0.19.9
replace (
k8s.io/code-generator => k8s.io/code-generator v0.19.9
github.com/kubeflow/mpi-operator/v2 => ./v2
)
106 changes: 106 additions & 0 deletions go.sum

Large diffs are not rendered by default.

32 changes: 24 additions & 8 deletions hack/python-sdk/gen-sdk.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,31 +21,47 @@ set -o pipefail
SWAGGER_JAR_URL="https://repo1.maven.org/maven2/org/openapitools/openapi-generator-cli/5.1.0/openapi-generator-cli-5.1.0.jar"
SWAGGER_CODEGEN_JAR="hack/python-sdk/openapi-generator-cli.jar"
SWAGGER_CODEGEN_CONF="hack/python-sdk/swagger_config.json"
SWAGGER_CODEGEN_FILE="pkg/apis/kubeflow/v1/swagger.json"
SWAGGER_V1_CODEGEN_FILE="pkg/apis/kubeflow/v1/swagger.json"
SWAGGER_V2_CODEGEN_FILE="v2/pkg/apis/kubeflow/v2beta1/swagger.json"
SDK_OUTPUT_PATH="sdk/python"

if [ -z "${GOPATH:-}" ]; then
export GOPATH=$(go env GOPATH)
fi

# Backup existing openapi_generated.go
# Backup existing v1 openapi_generated.go
mv pkg/apis/kubeflow/v1/openapi_generated.go pkg/apis/kubeflow/v1/openapi_generated.go.backup

echo "Generating OpenAPI specification ..."
echo "Generating V1 OpenAPI specification ..."
openapi-gen --input-dirs github.com/kubeflow/mpi-operator/pkg/apis/kubeflow/v1,github.com/kubeflow/common/pkg/apis/common/v1 --output-package github.com/kubeflow/mpi-operator/pkg/apis/kubeflow/v1 --go-header-file hack/boilerplate/boilerplate.go.txt

echo "Generating swagger file ..."
go run hack/python-sdk/main.go 0.1 > ${SWAGGER_CODEGEN_FILE}
echo "Generating V1 swagger file ..."
go run hack/python-sdk/main.go v1 > ${SWAGGER_V1_CODEGEN_FILE}

# Backup existing v2 openapi_generated.go
mv v2/pkg/apis/kubeflow/v2beta1/openapi_generated.go v2/pkg/apis/kubeflow/v2beta1/openapi_generated.go.backup

echo "Generating V2 OpenAPI specification ..."
openapi-gen --input-dirs github.com/kubeflow/mpi-operator/v2/pkg/apis/kubeflow/v2beta1,github.com/kubeflow/common/pkg/apis/common/v1 --output-package github.com/kubeflow/mpi-operator/v2/pkg/apis/kubeflow/v2beta1 --go-header-file hack/boilerplate/boilerplate.go.txt

echo "Generating V2 swagger file ..."
go run hack/python-sdk/main.go v2beta1 > ${SWAGGER_V2_CODEGEN_FILE}

echo "Downloading the swagger-codegen JAR package ..."
if ! [ -f ${SWAGGER_CODEGEN_JAR} ]; then
wget -O ${SWAGGER_CODEGEN_JAR} ${SWAGGER_JAR_URL}
fi

echo "Generating Python SDK for Kubeflow MPI-Operator ..."
java -jar ${SWAGGER_CODEGEN_JAR} generate -i ${SWAGGER_CODEGEN_FILE} -g python-legacy -o ${SDK_OUTPUT_PATH} -c ${SWAGGER_CODEGEN_CONF}
echo "Generating V1 Python SDK for Kubeflow MPI-Operator ..."
java -jar ${SWAGGER_CODEGEN_JAR} generate -i ${SWAGGER_V1_CODEGEN_FILE} -g python-legacy -o ${SDK_OUTPUT_PATH}/v1 -c ${SWAGGER_CODEGEN_CONF}

# Rollback the current openapi_generated.go
echo "Generating V2 Python SDK for Kubeflow MPI-Operator ..."
java -jar ${SWAGGER_CODEGEN_JAR} generate -i ${SWAGGER_V2_CODEGEN_FILE} -g python-legacy -o ${SDK_OUTPUT_PATH}/v2beta1 -c ${SWAGGER_CODEGEN_CONF}

# Rollback the current V1 openapi_generated.go
mv pkg/apis/kubeflow/v1/openapi_generated.go.backup pkg/apis/kubeflow/v1/openapi_generated.go

# Rollback the current V2 openapi_generated.go
mv v2/pkg/apis/kubeflow/v2beta1/openapi_generated.go.backup v2/pkg/apis/kubeflow/v2beta1/openapi_generated.go

echo "Kubeflow MPI-Operator Python SDK is generated successfully to folder ${SDK_OUTPUT_PATH}/."
20 changes: 14 additions & 6 deletions hack/python-sdk/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,25 +26,31 @@ import (
"k8s.io/klog"
"k8s.io/kube-openapi/pkg/common"

mpijob "github.com/kubeflow/mpi-operator/pkg/apis/kubeflow/v1"
mpijobv1 "github.com/kubeflow/mpi-operator/pkg/apis/kubeflow/v1"
mpijobv2 "github.com/kubeflow/mpi-operator/v2/pkg/apis/kubeflow/v2beta1"
)

// Generate OpenAPI spec definitions for MPIJob Resource
func main() {
if len(os.Args) <= 1 {
klog.Fatal("Supply a version")
klog.Fatal("Supply the MPIJob version")
}

version := os.Args[1]
if !strings.HasPrefix(version, "v") {
version = "v" + version
if version != "v1" && version != "v2beta1" {
fmt.Println("Only `v1` or `v2beta1` for MPIJob is supported now")
}

filter := func(name string) spec.Ref {
return spec.MustCreateRef(
"#/definitions/" + common.EscapeJsonPointer(swaggify(name)))
}

oAPIDefs := mpijob.GetOpenAPIDefinitions(filter)
var oAPIDefs map[string]common.OpenAPIDefinition
if version == "v1" {
oAPIDefs = mpijobv1.GetOpenAPIDefinitions(filter)
} else if version == "v2beta1" {
oAPIDefs = mpijobv2.GetOpenAPIDefinitions(filter)
}
defs := spec.Definitions{}
for defName, val := range oAPIDefs {
defs[swaggify(defName)] = val.Schema
Expand Down Expand Up @@ -72,10 +78,12 @@ func main() {

func swaggify(name string) string {
name = strings.Replace(name, "github.com/kubeflow/mpi-operator/pkg/apis/kubeflow/", "", -1)
name = strings.Replace(name, "github.com/kubeflow/mpi-operator/v2/pkg/apis/kubeflow/", "", -1)
name = strings.Replace(name, "github.com/kubeflow/common/pkg/apis/common/", "", -1)
name = strings.Replace(name, "github.com/kubernetes-sigs/kube-batch/pkg/client/clientset/", "", -1)
name = strings.Replace(name, "k8s.io/api/core/", "", -1)
name = strings.Replace(name, "k8s.io/apimachinery/pkg/apis/meta/", "", -1)
name = strings.Replace(name, "k8s.io/apimachinery/pkg/runtime/", "", -1)
name = strings.Replace(name, "k8s.io/apimachinery/pkg/api/", "", -1)
name = strings.Replace(name, "k8s.io/kubernetes/pkg/controller/", "", -1)
name = strings.Replace(name, "k8s.io/client-go/listers/core/", "", -1)
Expand Down
14 changes: 4 additions & 10 deletions pkg/apis/kubeflow/v1/openapi_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions pkg/apis/kubeflow/v1/swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"info": {
"description": "Python SDK for MPI-Operator",
"title": "mpijob",
"version": "v0.1"
"version": "v1"
},
"paths": {},
"definitions": {
Expand Down Expand Up @@ -81,11 +81,11 @@
"type": "object",
"properties": {
"apiVersion": {
"description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources",
"description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
"type": "string"
},
"kind": {
"description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
"description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
"type": "string"
},
"metadata": {
Expand All @@ -107,7 +107,7 @@
],
"properties": {
"apiVersion": {
"description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources",
"description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
"type": "string"
},
"items": {
Expand All @@ -117,7 +117,7 @@
}
},
"kind": {
"description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
"description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
"type": "string"
},
"metadata": {
Expand Down
1 change: 0 additions & 1 deletion pkg/apis/kubeflow/v1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion pkg/apis/kubeflow/v1/zz_generated.defaults.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 0 additions & 34 deletions sdk/python/setup.py

This file was deleted.

File renamed without changes.
33 changes: 33 additions & 0 deletions sdk/python/v1/.gitlab-ci.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions sdk/python/v1/.openapi-generator-ignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 36 additions & 0 deletions sdk/python/v1/.openapi-generator/FILES

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions sdk/python/v1/.openapi-generator/VERSION

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions sdk/python/v1/.travis.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading