[FEATURE]Integration registry - loading and filtering

[YANG-DB]

Previous Step
Next Step

Is your feature request related to a problem?
As part of the Integration Ux workflow, once the Integration plugin is loaded it should load all the available integrations that are bundled in the integration repo.

The API needed from the backend should be as follows:
Query:

GET _integration/repository?filter=type:Logs&category:web,html

Response:

{
  "name": "nginx",
  "version": {
    "integ": "0.1.0",
    "schema": "1.0.0",
    "resource": "^1.23.0"
  },
  "description": "Nginx HTTP server collector",
  "Information":"file:///.../schema/logs/info.html",
  "identification": "instrumentationScope.attributes.identification",
  "categories": [
    "web","http"
  ],
  "collection":[
    {
      "logs": [{
        "info": "access logs",
        "input_type":"logfile",
        "dataset":"nginx.access",
        "labels" :["nginx","access"],
        "schema": "file:///.../schema/logs/access.json"
      },
        {
          "info": "error logs",
          "input_type":"logfile",
          "labels" :["nginx","error"],
          "dataset":"nginx.error",
        "schema": "file:///.../schema/logs/error.json"
        }]
    },
    {
      "metrics": [{
        "info": "status metrics",
        "input_type":"metrics",
        "dataset":"nginx.status",
        "labels" :["nginx","status"],
        "schema": "file:///.../schema/logs/status.json"
      }]
    }
  ],
  "repo": {
    "github": "https://github.com/opensearch-project/observability/tree/main/integrarions/nginx"
  }
}

The integration object schema is supported by both B/E & F/E for display & query to the correct fields
Integration config schema

---

The backend responsibilities :

• scan the Integration folder (on-load)

  • In the future this can also be loaded from a remote publish location

• load into cache each integration config file

• allow filtering on the registry API

The frontend responsibilities :

• enable shared info (html, images) resources in a CDN
• allow filtering for the integrations

In addition the following API is also supported

GET _integration/repository/$name

This call results in returning the cached integration config json object
It would be used for display and route the F/E for fetching the relevant page assets

Once an integration was selected

This page will require the F/E to fetch multiple assets from different locations

• images
• repository url
• license url
• html
• json schema objects for that integration content

What solution would you like?
implement B/E :

• registry loading & cache mechanism
• registry API
• registry filter API

implement F/E :

• integrations list display
• integrations filter display
• integration panel display

Flow Diagram

What alternatives have you considered?
N/A

Do you have any additional context?

• https://github.com/opensearch-project/observability/tree/2.x/integrations/docs
• [RFC] Integrations Design OpenSearch-Dashboards#3412

The UX full happy flow for loading an integration

[derek-ho]

Currently outdated, closing it out