add catalog meta info in the index template

integrations/docs/API/swagger.yaml

@@ -0,0 +1,287 @@
+openapi: 3.0.0
+info:
+  description: This is the Integration & Catalog API
+  version: "1.0.0"
+  title: Integration API
+  license:
+    name: Apache 2.0
+    url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
+tags:
+  - name: admins
+    description: Secured Admin-only calls
+  - name: developers
+    description: Operations available to regular developers
+paths:
+  /catalog:
+    get:
+      tags:
+        - developers
+      summary: schema list of loaded catalogs
+      operationId: searchCatalog
+      description: |
+        By passing in the appropriate search attributes, you can search for
+        available catalogs in the system
+      parameters:
+        - in: query
+          name: searchString
+          description: pass an optional search string for looking up catalog components
+          required: false
+          schema:
+            type: string
+        - in: query
+          name: limit
+          description: maximum number of records to return
+          schema:
+            type: integer
+            format: int32
+            minimum: 0
+            maximum: 50
+      responses:
+        '200':
+          description: search results matching criteria
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: '../../../schema/system/catalog.schema'
+        '400':
+          description: bad input parameter
+    post:
+      tags:
+        - admins
+      summary: adds a catalog item
+      operationId: addCatalog
+      description: |
+        Adds a catalog item to the system, expecting the internal URLs to be accessible
+      requestBody:
+        description: Inventory item to add
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '../../../schema/system/catalog.schema'
+      responses:
+        '201':
+          description: item created
+        '400':
+          description: 'invalid input, object invalid'
+        '409':
+          description: an existing item already exists
+  /repository:
+    get:
+      tags:
+        - developers
+      summary: repository of available integrations
+      operationId: searchRepository
+      description: |
+        By passing in the appropriate search attributes, you can search for
+        available integration in the repository (available for loading into the store)
+      parameters:
+        - in: query
+          name: searchString
+          description: pass an optional search string for looking up available Integrations
+          required: false
+          schema:
+            type: string
+        - in: query
+          name: limit
+          description: maximum number of records to return
+          schema:
+            type: integer
+            format: int32
+            minimum: 0
+            maximum: 50
+      responses:
+        '200':
+          description: search results matching criteria
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: '../../../schema/system/integration.schema'
+        '400':
+          description: bad input parameter
+    post:
+      tags:
+        - admins
+      summary: adds an Integration item
+      operationId: addIntegration
+      description: |
+       Adds an Integration to the repository, expecting the internal URLs to be accessible
+      requestBody:
+        required: true
+        description: Inventory item to add
+        content:
+          application/json:
+            schema:
+              $ref: '../../../schema/system/integration.schema'
+      responses:
+        '201':
+          description: item created
+        '400':
+          description: 'invalid input, object invalid'
+        '409':
+          description: an existing item already exists
+  /store:
+    get:
+      tags:
+        - developers
+      summary: get all stored Integrations
+      operationId: searchStore
+      description: |
+        By passing in the appropriate search attributes, you can search for
+        available Integrations in the store
+      parameters:
+        - in: query
+          name: searchString
+          description: pass an optional search string for looking up Integration in the store
+          required: false
+          schema:
+            type: string
+        - in: query
+          name: limit
+          description: maximum number of records to return
+          schema:
+            type: integer
+            format: int32
+            minimum: 0
+            maximum: 50
+      responses:
+        '200':
+          description: search results matching criteria
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: '../../../schema/system/integration-instance.schema'
+        '400':
+          description: bad input parameter
+    post:
+      tags:
+        - admins
+      summary: adds an Integration item to store
+      operationId: storeIntegration
+      description: |
+       Adds an Integration to the system store, expecting the internal URLs to be accessible
+      requestBody:
+        required: true
+        description: Inventory item to add
+        content:
+          application/json:
+            schema:
+              $ref: '../../../schema/system/integration-instance.schema'
+      responses:
+        '201':
+          description: item created
+        '400':
+          description: 'invalid input, object invalid'
+        '409':
+          description: an existing item already exists
+  /store/{id}:
+    get:
+      tags:
+        - developers
+      summary: get stored Integration status
+      operationId: integrationStatus
+      description: |
+        Get the stored Integration's status
+      parameters:
+        - in: path
+          name: id
+          description: Integration instance Id
+          required: true
+          schema:
+            type: string
+      responses:
+        '200':
+          description: stored integration status
+          content:
+            application/json:
+              schema:
+                $ref: '../../../schema/system/integration-instance.schema'
+        '400':
+          description: bad input parameter
+  /store/{id}/validate:
+    post:
+      tags:
+        - developers
+      summary: validates an Integration
+      operationId: integrationValidate
+      description: |
+        validate this integration including all its assets
+      parameters:
+        - in: path
+          name: id
+          description: Integration instance Id
+          required: true
+          schema:
+            type: string
+      responses:
+        '200':
+          description: integration store status
+          content:
+            application/json:
+              schema:
+                $ref: '../../../schema/system/integration-instance.schema'
+        '400':
+          description: bad input parameter
+  /store/{id}/upload:
+    post:
+      tags:
+        - developers
+      summary: upload Integration's assets
+      operationId: integrationUploadAssets
+      description: |
+        Load a list of assets associated with this integration into 
+        available Integrations in the system store
+      requestBody:
+        required: false
+        description: Inventory items to upload
+        content:
+          application/json:
+            schema:
+              $ref: '../../../schema/system/integration-instance.schema'
+      parameters:
+        - in: path
+          name: id
+          description: Integration instance Id
+          required: true
+          schema:
+            type: string
+      responses:
+        '200':
+          description: integration store status
+          content:
+            application/json:
+              schema:
+                $ref: '../../../schema/system/integration-instance.schema'
+        '400':
+          description: bad input parameter
+  /store/{id}/activate:
+    put:
+      tags:
+        - developers
+      summary: activate an Integration
+      operationId: integrationActivate
+      description: |
+        attempts activating an integration that may be in an intermediate phase
+        during the storage process
+      parameters:
+        - in: path
+          name: id
+          description: Integration instance Id
+          required: true
+          schema:
+            type: string
+      responses:
+        '200':
+          description: integration store status
+          content:
+            application/json:
+              schema:
+                $ref: '../../../schema/system/integration-instance.schema'
+        '400':
+          description: bad input parameter

integrations/docs/Integration-API.md

@@ -8,7 +8,10 @@ Integrations are a stateful bundle which will be stored inside a system `.integr
 ### Flow Diagram
 ![Screenshot 2023-03-01 at 7 00 50 PM](https://user-images.githubusercontent.com/48943349/222320100-cac40749-9e5a-4e90-8ff2-386958adc06d.png)
 
-### Load Integrations Repositoty
+
+### [Swagger API reference](../docs/API/swagger.yaml)
+
+### Load Integrations Repository
 As part of the Integration Ux workflow, once the Integration plugin is loaded it should load all the available integrations that are bundled in the integration repo.
 
 ![Screenshot 2023-03-01 at 10 01 45 AM](https://user-images.githubusercontent.com/48943349/222223963-9c740f33-e156-4541-88cf-67d70131410f.png)
@@ -83,7 +86,7 @@ The frontend responsibilities :
 
 ![Screenshot 2023-03-01 at 10 13 41 AM](https://user-images.githubusercontent.com/48943349/222226930-1d9a684d-7f19-4aaf-b601-32bf7ce08920.png)
 
-In addition the following API is also supported
+In addition, the following API is also supported
 ```
 GET _integration/repository/$name
 ```
@@ -132,7 +135,7 @@ Store API:
 ```
 POST _integration/store/$instance_name
 ```
-The $instance_name represents the specific name the integration was instanciated with - for example an Nginx Integration can be a template for multiple Nginx instances
+The $instance_name represents the specific name the integration was instanced with - for example, Nginx Integration can be a template for multiple Nginx instances
 each representing different domain / aspect such as geographic.
 
 ```jsoon
@@ -190,7 +193,11 @@ If the user keeps all the original data-stream naming convention (namespace and
 In case the user wants to update the data-stream / index naming details - he may do so using dedicated window.
 Selection of the naming convention may also display available existing data-streams that are selectable if the user wants to choose from available ones and not creating new templates.
 
-Once user changes the data-stream / index pattern - this will be reflected in every asset that has this attribute. 
+Once user changes the data-stream / index pattern - this will be reflected in every asset that has this attribute.
+ - update the asset name (according to the `instance_name` field)
+   - `${instance_name}-assetName.json`, this can also be extended using more configurable patterns such as `${instance_name}-{dataset}-{domain}-assetName.json`
+ - update the index template's `index_pattern` field with the added pattern 
+   - "index_patterns":` ["sso_logs-*-*"]` -> `["sso_logs-*-*", "myLogs-*"]`
 
 #### Loading Integration
 
@@ -206,14 +213,13 @@ Each step may result on one of two result
 - `ready` - this will transition it to the next step
 - `maintenance` - this will hold until user fix issues
 
-Once the Integration instance was stored in the integration `store` index, it will have a `loading` status as displayed in the
-
-first image.
+Once the Integration instance was stored in the integration `store` index, it will have a `loading` status as displayed in the first image.
 
 Next the integration instance will undergo a validation phase in which
  - assets will be validated with the schema to match fields to the mapping
+ - assets containing index patterns will be validated any index with these pattern exists 
  - datasource will be validated to verify connection is accessible
- - mapping templates are verified to exist
+ - mapping templates are verified to exist 
 
 If any of the validation failed - the API (the call to `_integration/store/$instance_name` ) will return a status indicating 
 the current state of the integration:

integrations/docs/Integration-loading.md

@@ -11,8 +11,12 @@ The integrations passing this process can be available out of the box once the O
 Once an Observability is distributed, it is pre-bundled with the verified Integrations. These integrations are packaged in a dedicated folder.
 
 **Integration Lifecycle**
-![](img/integration-loading-lifecycle.png)
-
+```
+- LOADING
+    - VALIDATION
+        - UPLOAD
+            - READY
+``` 
 Observability bootstrap initiates the state for all the Integrations bundled with the distribution, the initial state is
 
 ***Loading*** - indicating the integration is still loading and has not yet been verified for runtime readiness.