5.0.1 (#59)

* 2024052802 - API v5.0 Updated JSON schema for Developer Portal definition * 2024052901 - API v5.0.1 Backstage.io integration - first commit * 20240530-01 - Backstage devportal * 20240530-01 - Backstage devportal * 20240612-01 - Backstage devportal * 20240613-01 - Backstage devportal, NGINX One SaaS console dev * 20240614 - backstage.io and NGINX One Cloud Console alpha stage support
f5devcentral · Jun 14, 2024 · 0d84cad · 0d84cad
1 parent 449c2ab
commit 0d84cad
Show file tree

Hide file tree

Showing 14 changed files with 1,187 additions and 51 deletions.
diff --git a/FEATURES.md b/FEATURES.md
@@ -30,9 +30,10 @@
 
 ### API Gateway - Developer Portal
 
-| Feature                                         | API v4.2 | API v5.0 | Notes                     |
-|-------------------------------------------------|----------|----------|---------------------------|
-| Developer Portal generation from OpenAPI schema | X        | X        | <li>Based on Redocly</li> |
+| Feature                             | API v4.2 | API v5.0  | Notes |
+|-------------------------------------|----------|-----------|-------|
+| Redocly-based developer portal      | X        | X         |       |
+| Backstage.io-based developer portal |          | X         |       |
 
 ### Client authentication
 

diff --git a/README.md b/README.md
@@ -2,22 +2,22 @@
 
 [![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)
 
-This project provides a set of declarative REST API for [NGINX Instance Manager](https://docs.nginx.com/nginx-management-suite/nim/).
+This project provides a set of declarative REST API for [NGINX Instance Manager](https://docs.nginx.com/nginx-management-suite/nim/) and [NGINX One Cloud Console - currently in early stage](https://docs.nginx.com/nginx-one/).
 
 It can be used to manage NGINX Plus configuration lifecycle and to create NGINX Plus configurations using JSON service definitions.
 
-GitOps integration is supported when used with NGINX Instance Manager: source of truth is checked for updates (NGINX App Protect policies, TLS certificates, keys and chains/bundles, Swagger/OpenAPI definitions) and NGINX configurations are automatically kept in sync.
+GitOps integration is supported: source of truth is checked for updates (NGINX App Protect policies, TLS certificates, keys and chains/bundles, Swagger/OpenAPI definitions) and NGINX configurations are automatically kept in sync.
 
 Use cases include:
 
 - Rapid configuration generation and templating
-- CI/CD integration with NGINX Instance Manager (instance groups and staged configs)
-- NGINX App Protect DevSecOps integration
+- CI/CD integration with NGINX Instance Manager (instance groups and staged configs) and NGINX One Cloud Console (clusters)
+- NGINX App Protect DevSecOps integration (NGINX Instance Manager only)
 - API Gateway deployments with automated Swagger / OpenAPI schema import
 - API Developer portals zero-touch deployment
 - GitOps integration with source of truth support for
   - NGINX App Protect WAF policies
-  - TLS certificates, keys and chains/bundles
+  - TLS certificates, keys and chains/bundles (NGINX Instance Manager only)
   - mTLS certificates
   - `http` snippets, upstreams, servers, locations
   - `stream` snippets, upstreams, servers
@@ -27,6 +27,7 @@ Use cases include:
 ## Supported releases
 
 - NGINX Instance Manager 2.14+
+- NGINX One Cloud Console
 - NGINX Plus R30+
 - NGINX App Protect WAF 4.8+
 

diff --git a/USAGE-v5.0.md b/USAGE-v5.0.md
@@ -37,6 +37,13 @@ The JSON schema is self explanatory. See also the [sample Postman collection](/c
       - `.output.nms.policies[].versions[].displayName` the policy version's display name
       - `.output.nms.policies[].versions[].description` the policy version's description
       - `.output.nms.policies[].versions[].contents` this can be either base64-encoded or be a HTTP(S) URL that will be fetched dynamically from a source of truth
+  - *nginxone* - NGINX configuration is published to a NGINX One Cloud Console cluster
+    - `.output.nginxone.url` the NGINX One Cloud Console URL
+    - `.output.nginxone.namespace` the NGINX One Cloud Console namespace
+    - `.output.nginxone.token` the authentication token
+    - `.output.nginxone.cluster` the cluster name
+    - `.output.nginxone.synctime` **optional**, used for GitOps autosync. When specified and the declaration includes HTTP(S) references to NGINX App Protect policies, TLS certificates/keys/chains, the HTTP(S) endpoints will be checked every `synctime` seconds and if external contents have changed, the updated configuration will automatically be published to NGINX Instance Manager
+    - `.output.nginxone.modules` an optional array of NGINX module names (ie. 'ngx_http_app_protect_module', 'ngx_http_js_module','ngx_stream_js_module')
 - `.declaration` describes the NGINX configuration to be created.
 
 ### Locations ###
@@ -111,8 +118,9 @@ Declaration path `.declaration.http.servers[].locations[].apigateway` defines th
 - `api_gateway.strip_uri` - removes the `.declaration.http.servers[].locations[].uri` part of the URI before forwarding requests to the upstream
 - `api_gateway.server_url` - the base URL of the upstream server
 - `developer_portal.enabled` - enable/disable Developer portal provisioning
-- `developer_portal.type` - developer portal type. Currently supported are: `redocly`
-- `developer_portal.redocly.uri` - the trailing part of the Developer portal URI, this is appended to `.declaration.http.servers[].locations[].uri`. If omitted it defaults to `devportal.html`
+- `developer_portal.type` - developer portal type. Currently supported are: `redocly`, `backstage`
+- `developer_portal.redocly.*` - Redocly-based developer portal parameters. See the [Postman collection](/contrib/postman)
+- `developer_portal.backstage.*` - Backstage-based developer portal parameters. See the [Postman collection](/contrib/postman)
 - `authentication` - optional, used to enforce authentication at the API Gateway level
 - `authentication.client[]` - authentication profile names
 - `authentication.enforceOnPaths` - if set to `true` authentication is enforced on all API endpoints listed under `authentication.paths`. if set to `false` authentication is enforced on all API endpoints but those listed under `authentication.paths`
@@ -124,13 +132,15 @@ Declaration path `.declaration.http.servers[].locations[].apigateway` defines th
 - `rate_limit` - optional, used to enforce rate limiting at the API Gateway level
 - `rate_limit.enforceOnPaths` - if set to `true` rate limiting is enforced on all API endpoints listed under `rate_limit.paths`. if set to `false` rate limiting is enforced on all API endpoints but those listed under `rate_limit.paths`
 
-A sample API Gateway declaration to publish the `https://petstore.swagger.io` REST API and enforce:
+A sample API Gateway declaration to publish the `https://petstore.swagger.io` REST API using:
 
 - REST API endpoint URIs
 - HTTP Methods
 - Rate limiting on `/user/login`, `/usr/logout` and `/pet/{petId}/uploadImage`
 - JWT authentication on `/user/login`, `/usr/logout` and `/pet/{petId}/uploadImage`
 - JWT claim-based authorization on `/user/login`, `/usr/logout` and `/pet/{petId}/uploadImage`
+- Redocly-based developer portal
+- NGINX App Protect WAF security
 
 can be found in the [Postman collection](/contrib/)
 
@@ -195,4 +205,4 @@ For a list of all supported authentication profile types see the [feature matrix
 
 ### Usage Examples ###
 
-A sample Postman collection is available [here](/contrib/postman)
+A sample Postman collection is available [here](/contrib/postman)
diff --git a/contrib/postman/NGINX Declarative API.postman_collection.json b/contrib/postman/NGINX Declarative API.postman_collection.json
diff --git a/contrib/postman/README.md b/contrib/postman/README.md
@@ -4,7 +4,7 @@ This collection contains:
 
 API v5.0 - Latest
 - `Configuration generation` - Declaration examples with output to plaintext, JSON, Kubernetes ConfigMap, HTTP POST
-- `Declarative automation examples` - Several examples and use cases
+- `Declarative automation examples` - Several examples and use cases for NGINX Instance Manager and NGINX One Cloud Console
   - `API Gateway` - Sample API gateway requests for Swagger and OpenAPI schemas import
   - `CRUD automation` - Sample requests for CRUD-based automation
   - `GitOps autosync` - GitOps automation demo

diff --git a/etc/config.toml b/etc/config.toml
@@ -22,6 +22,8 @@ auth_server_root = "authn/server"
 
 authz_client_root = "authz/client"
 
+devportal_root = "devportal"
+
 # NGINX Declarative API Server
 [apiserver]
 host = "0.0.0.0"

diff --git a/src/V5_0_CreateConfig.py b/src/V5_0_CreateConfig.py
@@ -66,6 +66,9 @@ def createconfig(declaration: ConfigDeclaration, apiversion: str, runfromautosyn
     # NGINX auxiliary files for staged config
     auxFiles = {'files': []}
 
+    # Extra manifests to be returned to the caller
+    extraOutputManifests = []
+
     try:
         # Pydantic JSON validation
         ConfigDeclaration(**declaration.model_dump())
@@ -400,7 +403,7 @@ def createconfig(declaration: ConfigDeclaration, apiversion: str, runfromautosyn
                                         {"code": status,
                                          "content": f"invalid server authentication profile [{openApiAuthProfile[0]['profile']}] for OpenAPI schema [{loc['apigateway']['openapi_schema']['content']}]"}}}
 
-                        status, apiGatewayConfigDeclaration = v5_0.APIGateway.createAPIGateway(locationDeclaration = loc, authProfiles = d['declaration']['http']['authentication'])
+                        status, apiGatewayConfigDeclaration, openAPISchemaJSON = v5_0.APIGateway.createAPIGateway(locationDeclaration = loc, authProfiles = loc['apigateway']['openapi_schema']['authentication'])
 
                         # API Gateway configuration template rendering
                         if apiGatewayConfigDeclaration:
@@ -415,21 +418,35 @@ def createconfig(declaration: ConfigDeclaration, apiversion: str, runfromautosyn
                     # API Gateway Developer portal provisioning
                     if loc['apigateway'] and loc['apigateway']['developer_portal'] and 'enabled' in loc['apigateway']['developer_portal'] and loc['apigateway']['developer_portal']['enabled'] == True:
 
-                        status, devPortalHTML = v5_0.DevPortal.createDevPortal(locationDeclaration = loc, authProfiles = d['declaration']['http']['authentication'])
-
-                        if status != 200:
-                            return {"status_code": 412,
-                                    "message": {"status_code": status, "message":
-                                        {"code": status, "content": f"Developer Portal creation failed for {loc['uri']}"}}}
-
-                        ### Add optional API Developer portal HTML files
+                        ### Redocly developer portal - Add optional API Developer portal HTML files
                         # devPortalHTML
-                        if loc['apigateway']['developer_portal']['type'].lower() == "redocly":
+                        if loc['apigateway']['developer_portal']['type'].lower() == 'redocly':
+                            status, devPortalHTML = v5_0.DevPortal.createDevPortal(locationDeclaration=loc,
+                                                                                   authProfiles=
+                                                                                   d['declaration']['http'][
+                                                                                       'authentication'])
+
+                            if status != 200:
+                                return {"status_code": 412,
+                                        "message": {"status_code": status, "message":
+                                            {"code": status,
+                                             "content": f"Developer Portal creation failed for {loc['uri']}"}}}
+
                             newAuxFile = {'contents': devPortalHTML, 'name': NcgConfig.config['nms']['devportal_dir'] +
                                                                                loc['apigateway']['developer_portal']['redocly']['uri']}
                             auxFiles['files'].append(newAuxFile)
 
-                        ### / Add optional API Developer portal HTML files
+                        ### / Redocly developer portal - Add optional API Developer portal HTML files
+
+                        ### Backstage developer portal - Create Kubernetes Backstage manifest
+                        # devPortalHTML
+                        if loc['apigateway']['developer_portal']['type'].lower() == 'backstage':
+                            backstageManifest = j2_env.get_template(f"{NcgConfig.config['templates']['devportal_root']}/backstage.tmpl").render(
+                                declaration=loc['apigateway']['developer_portal']['backstage'], openAPISchema = v5_0.MiscUtils.json_to_yaml(openAPISchemaJSON), ncgconfig=NcgConfig.config)
+
+                            extraOutputManifests.append(backstageManifest)
+
+                        ### / Backstage developer portal - Create Kubernetes Backstage manifest
 
                     if loc['rate_limit'] is not None:
                         if 'profile' in loc['rate_limit'] and loc['rate_limit']['profile'] and loc['rate_limit'][
@@ -547,30 +564,30 @@ def createconfig(declaration: ConfigDeclaration, apiversion: str, runfromautosyn
         # NGINX auxiliary files for staged config
         auxFiles['rootDir'] = NcgConfig.config['nms']['config_dir']
 
-        return v5_0.NMSOutput.NMSOutput(d = d, declaration = declaration, apiversion = apiversion,
+        finalReply = v5_0.NMSOutput.NMSOutput(d = d, declaration = declaration, apiversion = apiversion,
                                  b64HttpConf = b64HttpConf, b64StreamConf = b64StreamConf,
                                  configFiles = configFiles,
                                  auxFiles = auxFiles,
                                  runfromautosync = runfromautosync, configUid = configUid )
 
+        finalReply['message']['message']['content']['manifests'] = extraOutputManifests
+
+        return finalReply
+
     elif decltype.lower() == 'nginxone':
         # Output to NGINX One SaaS Console
 
         # NGINX configuration files for staged config
         configFiles['name'] = NcgConfig.config['nms']['config_dir']
 
         # NGINX auxiliary files for staged config
-        # TODO
-        # auxFiles['name'] = NcgConfig.config['nms']['config_dir']
-
-        #return v5_0.NGINXOneOutput.NGINXOneOutput(d = d, declaration = declaration, apiversion = apiversion,
-        #                         b64HttpConf = b64HttpConf, b64StreamConf = b64StreamConf,
-        #                         configFiles = configFiles,
-        #                         auxFiles = auxFiles,
-        #                         runfromautosync = runfromautosync, configUid = configUid )
-
-        return {"status_code": 501, "message": {"code": 501, "content": "NGINX One support not yet available"}}
+        auxFiles['name'] = NcgConfig.config['nms']['config_dir']
 
+        return v5_0.NGINXOneOutput.NGINXOneOutput(d = d, declaration = declaration, apiversion = apiversion,
+                                 b64HttpConf = b64HttpConf, b64StreamConf = b64StreamConf,
+                                 configFiles = configFiles,
+                                 auxFiles = auxFiles,
+                                 runfromautosync = runfromautosync, configUid = configUid )
     else:
         return {"status_code": 422, "message": {"status_code": 422, "message": f"output type {decltype} unknown"}}
 

diff --git a/src/V5_0_NginxConfigDeclaration.py b/src/V5_0_NginxConfigDeclaration.py
@@ -772,16 +772,35 @@ class DevPortal_Redocly(BaseModel, extra="forbid"):
     uri: Optional[str] = "/devportal.html"
 
 
+class DevPortal_Backstage(BaseModel, extra="forbid"):
+    name: str
+    lifecycle: Optional[str] = "production"
+    owner: str = ""
+    system: Optional[str] = ""
+
+    @model_validator(mode='after')
+    def check_type(self) -> 'DevPortal_Backstage':
+        _lifecycle = self.lifecycle
+
+        valid = ['experimental', 'production', 'deprecated']
+        if _lifecycle not in valid:
+            raise ValueError(f"Invalid developer portal type [{_lifecycle}] must be one of {str(valid)}")
+
+        return self
+
+
 class DeveloperPortal(BaseModel, extra="forbid"):
     enabled: Optional[bool] = False
     type: str
     redocly: Optional[DevPortal_Redocly] = {}
+    backstage: Optional[DevPortal_Backstage] = {}
 
     @model_validator(mode='after')
     def check_type(self) -> 'DeveloperPortal':
-        _type, _redocly = self.type, self.redocly
+        _type, _redocly, _backstage = self.type, self.redocly, self.backstage
+
+        valid = ['redocly', 'backstage']
 
-        valid = ['redocly']
         if _type not in valid:
             raise ValueError(f"Invalid developer portal type [{_type}] must be one of {str(valid)}")
 
@@ -790,6 +809,9 @@ def check_type(self) -> 'DeveloperPortal':
         if _type == 'redocly' and not _redocly:
             isError = True
 
+        if _type == 'backstage' and not _backstage:
+            isError = True
+
         if isError:
             raise ValueError(f"Missing developer portal data for type [{_type}]")
 

diff --git a/src/v5_0/APIGateway.py b/src/v5_0/APIGateway.py
@@ -34,4 +34,4 @@ def createAPIGateway(locationDeclaration: dict, authProfiles: Authentication={})
         apiGwDeclaration['paths'] = apiSchema.paths()
         apiGwDeclaration['version'] = apiSchema.version()
 
-    return 200, apiGwDeclaration
+    return 200, apiGwDeclaration, apiSchemaString['content']
diff --git a/src/v5_0/MiscUtils.py b/src/v5_0/MiscUtils.py
@@ -46,6 +46,13 @@ def yaml_to_json(document: str):
     return json.dumps(yaml.safe_load(document))
 
 
+"""
+JSON TO YAML conversion
+"""
+def json_to_yaml(document: str):
+    return yaml.dump(json.loads(document))
+
+
 """
 Returns a unique ID
 """