Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(LH-89186): Write maintainable CLI tool for API doc and SDK generation #62

Merged
merged 47 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
1b4dbcd
feat(lh-88296): add new Golang CLI to generate OpenAPI docs, which is…
siddhuwarrier Jan 7, 2025
c256fa6
WIP
siddhuwarrier Jan 7, 2025
edfb1e8
Fix gitignore
siddhuwarrier Jan 7, 2025
646ea5b
Read config from github
siddhuwarrier Jan 7, 2025
add9f8a
Fix config file URL
siddhuwarrier Jan 7, 2025
3215034
fix(lh-89186): formatting
siddhuwarrier Jan 7, 2025
56b535d
Add object service and transaction service to config
siddhuwarrier Jan 7, 2025
3fd47ec
More commits
siddhuwarrier Jan 8, 2025
876f65d
restore myenv to make PR reviewable
siddhuwarrier Jan 8, 2025
74b7749
remove duplicate config.yaml
siddhuwarrier Jan 8, 2025
25967c5
WIP
siddhuwarrier Jan 8, 2025
b611a15
Merge branch 'main' into LH-89186-improve-api-doc-generation
siddhuwarrier Jan 9, 2025
2794ea4
fix(lh-89186): add python SDK generation and publishing support
siddhuwarrier Jan 10, 2025
6fa86e9
test(lh-89186): add test, Jenkinsfile
siddhuwarrier Jan 10, 2025
92afca8
Switch to using Github actions because we used main instead of master…
siddhuwarrier Jan 10, 2025
532975d
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
4a2e94e
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
3e27293
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
290aba5
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
6493b69
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
9ba6aea
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
1e0ad58
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
d806de7
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
1f30444
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
558b112
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
a370186
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
2395f09
release separately
siddhuwarrier Jan 10, 2025
55dcb97
release separately
siddhuwarrier Jan 10, 2025
8776a56
release separately
siddhuwarrier Jan 10, 2025
e25d31e
release separately
siddhuwarrier Jan 10, 2025
224c5c8
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
ff21bb3
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
32d3df3
burak-crush-pineapple
siddhuwarrier Jan 10, 2025
0a9c257
fix CI
siddhuwarrier Jan 10, 2025
aaa5b80
Add more info
siddhuwarrier Jan 10, 2025
b3fb687
feat(lh-89186): add support for building debian packages
siddhuwarrier Jan 10, 2025
9bd9229
fix(lh-89186): do not tag and release unless branch is main
siddhuwarrier Jan 11, 2025
53d485c
fix(lh-89186): support install-pre-reqs on non-Mac devices
siddhuwarrier Jan 12, 2025
27de6fd
Don't allow install-pre-reqs for Linux
siddhuwarrier Jan 12, 2025
9eda220
add new command to generate postman collection
siddhuwarrier Jan 12, 2025
367ba35
add new command to generate postman collection
siddhuwarrier Jan 12, 2025
76de22e
add new command to generate postman collection
siddhuwarrier Jan 13, 2025
2b258e5
fix(lh-89186): support using local openapi-generator-cli installation…
siddhuwarrier Jan 13, 2025
0f9cd5b
test(lh-89186): add more unit tests
siddhuwarrier Jan 13, 2025
0a27a27
fix(lh-89186): fix CODEOWNERS
siddhuwarrier Jan 14, 2025
9f0e9e8
fix CI
siddhuwarrier Jan 14, 2025
2f5be93
fix CI
siddhuwarrier Jan 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .DS_Store
Binary file not shown.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @cisco-lockhart/cdo-platform-team @cisco-lockhart/public-api-approvers
87 changes: 87 additions & 0 deletions .github/workflows/golang.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: build OpenAPI docs generation tool
on:
push:
branches: [ master, main ]
paths:
- scripts/golang/**
pull_request:
branches: [ master, main ]
paths:
- scripts/golang/**
jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-tags: true

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23'

- name: Install dependencies
working-directory: scripts/golang
run: |
go mod tidy
go install github.com/onsi/ginkgo/v2/ginkgo@latest

- name: Run tests
working-directory: scripts/golang
run: |
ginkgo -r -v

- name: Build Snapshots using Goreleaser
if: github.ref != 'refs/heads/main'
uses: goreleaser/goreleaser-action@v6
with:
args: release --snapshot
workdir: 'scripts/golang'

- name: Archive Snapshots
if: github.ref != 'refs/heads/main'
uses: actions/upload-artifact@v4
with:
name: dist-snapshots
path: scripts/golang/dist/*.tar.gz

# when branch is main, release a new version
- name: Bump version and push tag
id: tag_version
if: github.ref == 'refs/heads/main'
uses: mathieudutour/github-tag-action@v6.2
with:
github_token: ${{ secrets.GITHUB_TOKEN }}

release:
runs-on: ubuntu-latest
needs: build
if: github.ref == 'refs/heads/main'

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-tags: true
fetch-depth: 0

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23'

- name: Build using Goreleaser
uses: goreleaser/goreleaser-action@v6
with:
args: release
workdir: 'scripts/golang'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- uses: jfrog/setup-jfrog-cli@v4
env:
JF_URL: ${{ vars.ARTIFACTORY_URL }}
JF_ACCESS_TOKEN: ${{ secrets.ARTIFACTORY_ACCESS_TOKEN }}
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
node_modules/
.DS_Store
**/.idea
**/.DS_Store
scripts/golang/sdks/
scripts/golang/openapitools.json
**/cloud-fw-mgr-api-docs
46 changes: 46 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# How to Contribute

Thank you for your interest in contributing to this repository. If you are not a Cisco employee, we
unfortunately cannot accept pull request submissions from you. However, we welcome you to open issues.

If you are a Cisco employee who is working on a microservice that will become part of this repository, read on.

# Adding a new microservice

- Edit `cloud-fw-mgr-api-docs.config.yaml` in the root of this repo, and add the new microservice to
the `services` section. For example:

```yaml
- name: "my-new-service"
url: "https://staging.manage.security.cisco.com/api/platform/my-new-service/v3/api-docs.yaml"
```
- Edit `api-changelog.md` to add a short description of what APIs your new microservice provides.
- Submit a PR (please see commit message guidelines).
- You will need approval from the CODEOWNERS. See `.github/CODEOWNERS` to see who to request
approval from.
- Once your PR is approved, you will be (Cisco network only) be able to view the changes at
[https://devnetapps.cisco.com/docs/cisco-security-cloud-control/](https://devnetapps.cisco.com/docs/cisco-security-cloud-control/).
- To get it published to [https://developer.cisco.com/docs/cisco-security-cloud-control](https://developer.cisco.com/docs/cisco-security-cloud-control),
please work with the Cisco Devnet team or [Siddhu Warrier](mailto:siwarrie@cisco.com), as this is a manual process that will involve reviewing the APIs you have added to ensure
that they meet Cisco style guidelines.

# Making API changes to an existing microservice

## My change is backwards-incompatible

To adhere to Cisco's policy around backwards compatibility, you **cannot** make backwards-incompatible changes to an existing microservice.
If you need to make a backwards-incompatible change, please work with [Jeremy Street](mailto:jerestre@cisco.com) and [Siddhu Warrier](mailto:siwarrie@cisco.com).

## My change is backwards-compatible

This can involve adding new APIs, or changing existing in APIs in a backwards-compatible manner.

- Edit `api-changelog.md` to add a short description of what APIs your new microservice provides.
- Submit a PR (please see commit message guidelines).
- You will need approval from the CODEOWNERS. See `.github/CODEOWNERS` to see who to request
approval from.
- Once your PR is approved, you will be (Cisco network only) be able to view the changes at
[https://devnetapps.cisco.com/docs/cisco-security-cloud-control/](https://devnetapps.cisco.com/docs/cisco-security-cloud-control/).
- To get it published to [https://developer.cisco.com/docs/cisco-security-cloud-control](https://developer.cisco.com/docs/cisco-security-cloud-control),
please work with the Cisco Devnet team or [Siddhu Warrier](mailto:siwarrie@cisco.com), as this is a manual process that will involve reviewing the APIs you have added to ensure
that they meet Cisco style guidelines.
87 changes: 86 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,93 @@
# Welcome to the Cisco Security Cloud Control Public API Documentation

These docs are published to DevNet using PubHub.
These docs are published to DevNet using PubHub.

See [https://pubhub.cisco.com/detail/5082] or contact siwarrie@cisco.com for details

## View your changes

Go to [https://devnetapps.cisco.com/docs/cisco-defense-orchestrator/].

# Contributions

- PRs with changes to this file will only accepted from contributors who work for Cisco, who use
a Github account associated with their Cisco email address to create the PR.

## Adding a new microservice

- Edit `cloud-fw-mgr-api-docs.config.yaml` in the root of this repo, and add the new microservice to
the `services` section. For example:

```yaml
- name: "my-new-service"
url: "https://staging.manage.security.cisco.com/api/platform/my-new-service/v3/api-docs.yaml"
```
- Edit `api-changelog.md` to add a short description of what APIs your new microservice provides.
- Submit a PR (please see commit message guidelines).
- You will need approval from the CODEOWNERS. See `.github/CODEOWNERS` to see who to request
approval from.
- Once your PR is approved, you will be (Cisco network only) be able to view the changes at
[https://devnetapps.cisco.com/docs/cisco-security-cloud-control/](https://devnetapps.cisco.com/docs/cisco-security-cloud-control/).
- To get it published to [https://developer.cisco.com/docs/cisco-security-cloud-control](https://developer.cisco.com/docs/cisco-security-cloud-control),
please work with the Cisco Devnet team or [Siddhu Warrier](mailto:siwarrie@cisco.com), as this is a manual process that will involve reviewing the APIs you have added to ensure
that they meet Cisco style guidelines.

## Making API changes to an existing microservice

### My change is backwards-incompatible

To adhere to Cisco's policy around backwards compatibility, you **cannot** make backwards-incompatible changes to an existing microservice.
If you need to make a backwards-incompatible change, please work with [Jeremy Street](mailto:jerestre@cisco.com) and [Siddhu Warrier](mailto:siwarrie@cisco.com).

### My change is backwards-compatible

This can involve adding new APIs, or changing existing in APIs in a backwards-compatible manner.

- Edit `api-changelog.md` to add a short description of what APIs your new microservice provides.
- Submit a PR (please see commit message guidelines).
- You will need approval from the CODEOWNERS. See `.github/CODEOWNERS` to see who to request
approval from.
- Once your PR is approved, you will be (Cisco network only) be able to view the changes at
[https://devnetapps.cisco.com/docs/cisco-security-cloud-control/](https://devnetapps.cisco.com/docs/cisco-security-cloud-control/).
- To get it published to [https://developer.cisco.com/docs/cisco-security-cloud-control](https://developer.cisco.com/docs/cisco-security-cloud-control),
please work with the Cisco Devnet team or [Siddhu Warrier](mailto:siwarrie@cisco.com), as this is a manual process that will involve reviewing the APIs you have added to ensure
that they meet Cisco style guidelines.

## PR guidelines

All PR commits should follow
the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) format.
This will help us automate the release process, and we will set the version of the CLI on the basis
of the commit made.

# The Golang CLI for API documentation generation

## Pre-requisites

- Install Golang: https://golang.org/doc/install
- Install cobra-cli: `go install github.com/spf13/cobra-cli@latest`
- Install pre-requisites:

```shell
go run main.go install-pre-reqs
```

## Development

### Add a new command

```shell
cobra-cli add <command-name>
```

### Testing

All tests are written using [Ginkgo](https://github.com/onsi/ginkgo)
and [Gomega](https://github.com/onsi/gomega).

To run your tests, use the following command:

```shell
cd /path/to/scripts/golang
ginkgo -r
```
40 changes: 40 additions & 0 deletions cloud-fw-mgr-api-docs.config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
services:
- name: "api-facade"
url: "https://staging.manage.security.cisco.com/api/platform/public-api/v3/api-docs.yaml"
- name: "msp-api"
url: "https://staging.manage.security.cisco.com/api/platform/msp-api/v3/api-docs.yaml"
- name: "device-upgrade"
url: "https://staging.manage.security.cisco.com/api/platform/device-upgrade/v3/api-docs.yaml"
- name: "object-service"
url: "https://staging.manage.security.cisco.com/api/platform/object-service/v3/api-docs.yaml"
- name: "transaction-service"
url: "https://staging.manage.security.cisco.com/api/platform/transaction-service/v3/api-docs.yaml"
info:
title: "Cisco Security Cloud Control API"
version: "1.5.0"
description: "Use the documentation to explore the endpoints Security Cloud Control has to offer"
contact:
name: "Cisco Security Cloud Control TAC"
email: "cdo.tac@cisco.com"
servers:
- url: https://edge.us.cdo.cisco.com/api/rest
description: US
- url: https://edge.eu.cdo.cisco.com/api/rest
description: EU
- url: https://edge.apj.cdo.cisco.com/api/rest
description: APJ
- url: https://edge.aus.cdo.cisco.com/api/rest
description: AUS
- url: https://edge.in.cdo.cisco.com/api/rest
description: IN
- url: https://edge.staging.cdo.cisco.com/api/rest
description: Staging
- url: https://edge.scale.cdo.cisco.com/api/rest
description: Scale
- url: https://edge.ci.cdo.cisco.com/api/rest
description: CI
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
2 changes: 2 additions & 0 deletions scripts/golang/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

dist/
60 changes: 60 additions & 0 deletions scripts/golang/.goreleaser.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# This is an example .goreleaser.yml file with some sensible defaults.
# Make sure to check the documentation at https://goreleaser.com

# The lines below are called `modelines`. See `:help modeline`
# Feel free to remove those if you don't want/need to use them.
# yaml-language-server: $schema=https://goreleaser.com/static/schema.json
# vim: set ts=2 sw=2 tw=0 fo=cnqoj

version: 1

before:
hooks:
# You may remove this if you don't use go modules.
- go mod tidy
# you may remove this if you don't need go generate
- go generate ./...

builds:
- env:
- CGO_ENABLED=0
goos:
- linux
- darwin
binary: cloud-fw-mgr-api-docs

archives:
- format: tar.gz
# this name template makes the OS and Arch compatible with the results of `uname`.
name_template: >-
cloud-fw-mgr-api-docs_
{{- title .Os }}_
{{- if eq .Arch "amd64" }}x86_64
{{- else if eq .Arch "386" }}i386
{{- else }}{{ .Arch }}{{ end }}
{{- if .Arm }}v{{ .Arm }}{{ end }}
# use zip for windows archives
format_overrides:
- goos: windows
format: zip

changelog:
sort: asc
filters:
exclude:
- "^docs:"
- "^test:"

nfpms:
-
id: cloud-fw-mgr-api-docs
package_name: cloud-fw-mgr-api-docs
section: default
priority: extra
maintainer: "CDO Ops <cdo.ops@cisco.com>"
formats:
- deb
description: |
Cloud Firewall Manager API Docs
vendor: "Cisco Systems"
homepage: "https://www.cisco.com"
Empty file added scripts/golang/LICENSE
Empty file.
Loading
Loading