Observability on ONEST network

[vrayulu]

Observability on the ONEST network provides the ability to understand and gain insights into the inner workings of the network by observing its behavior, performance, and state. Observability provides a range of benefits, including but not limited to:

1. Debugging & Troubleshooting: Observability provides the necessary data and insights to identify & diagnose issues, and determine their root causes. This is crucial for resolving bugs, performance bottlenecks, and other operational problems quickly.
2. Compliance & Auditing: Observability is essential for maintaining compliance with the ONEST protocol, specifications & policies. It provides the necessary data for auditing and reporting purposes.
3. Data-driven Services: Observability provides data-driven insights that can help ONEST evolve its services to help network and the network participants make informed choices based on real-time and historical data.
4. Security: Observability helps in detecting and responding to security threats. Unusual patterns or anomalies in data can be indicators of potential security breaches or threats.

The observability infrastructure within ONEST is designed to offer the following key features:

• Ingest multiple data sets: functional data (API payloads), network infra data (infra & api metrics) and registry data from both network participants and the network facilitator systems
• Process the incoming data: validate for protocol compliance and mask any PII information present in the input data
• Query the processed data via multiple modes: query API, dashboards (pre-created & ability to create new), and download the ingested data as files from cloud storage

The observability infrastructure and services will be accessible to all network participants, allowing them to harness these resources for various purposes, in accordance with the policies set forth by the ONEST network.

[vrayulu]

For the ONEST WIN Room Zero on 14-15 Sept 2023, we have created an observability sandbox that is available for both participants and the network. This sandbox allows you to submit data and visualize it through interactive dashboards. Please find below the API details to be used during the WIN Room Zero event for pushing the data:

API curl command:
use the API request path and API key provided in this sample for the WIN Room Zero event

curl --location 'https://dev.obsrv.sunbird.org/obsrv/v1/data/onest' \
--header 'Content-Type: application/json' \
--header 'x-api-key: TaXev840SX2K51Im3wbkT2g45kovddTx3lQbqtKy' \
--data '{
    "data": {
        "id": "onest-batch-1",
        "events": [{}]
    }
}'

Sample API Payloads

Please note the additional field “sync_response” in each event, this can be used to send the ACK/NACK or any other HTTP response received synchronously for the APIs.

curl --location 'https://dev.obsrv.sunbird.org/obsrv/v1/data/onest' \
--header 'Content-Type: application/json' \
--header 'x-api-key: TaXev840SX2K51Im3wbkT2g45kovddTx3lQbqtKy' \
--data '{
    "data": {
        "id": "onest-batch-1",
        "events": [
            {
                "context": {
                    "domain": "onest:courses",
                    "transaction_id": "a9aaecca-10b7-4d19-b640-b047a7c62196",
                    "message_id": "bb579fb8-cb82-4824-be12-fcbc405b6608",
                    "action": "search",
                    "timestamp": "2022-12-15T05:23:03.443Z",
                    "version": "1.1.0",
                    "bap_uri": "https://onest-network.becknprotocol.io/",
                    "bap_id": "onest.becknprotocol.io",
                    "ttl": "PT10M"
                },
                "message": {
                    "intent": {
                        "item": {
                            "descriptor": {
                                "name": "english"
                            }
                        }
                    }
                },
                "sync_response": {
                    "message": {
                        "ack": {
                            "status": "ACK"
                        }
                    }
                }
            }
        ]
    }
}'

curl --location 'https://dev.obsrv.sunbird.org/obsrv/v1/data/onest' \
--header 'Content-Type: application/json' \
--header 'x-api-key: TaXev840SX2K51Im3wbkT2g45kovddTx3lQbqtKy' \
--data '{
    "data": {
        "id": "onest-batch-1",
        "events": [
            {
                "context": {
                    "domain": "onest:courses",
                    "version": "1.1.0",
                    "action": "on_search",
                    "bap_id": "onest.becknprotocol.io",
                    "bap_uri": "https://onest-network.becknprotocol.io/",
                    "transaction_id": "a9aaecca-10b7-4d19-b640-b047a7c62196",
                    "message_id": "7db91181-718f-4720-83de-88e36e9f854e",
                    "ttl": "PT10M",
                    "timestamp": "2023-02-22T10:30:18.145Z",
                    "bpp_id": "infosys.springboard.io",
                    "bpp_uri": "https://infosys.springboard.io"
                },
                "message": {
                    "catalog": {
                        "descriptor": {
                            "name": "Catalog for English courses"
                        },
                        "providers": []
                    }
                },
                "sync_response": {
                    "message": {
                        "ack": {
                            "status": "ACK"
                        }
                    }
                }
            }
        ]
    }
}'

[vrayulu]

The ONEST network dashboards will be made accessible to all participants of the WIN Room by the conclusion of Day 1 of the event.

The dashboards will contain API metrics, Functional metrics, Insights & Operational metrics similar to the ONDC demo dashboard.

ONDC Demo Dashboard details:
URL - https://ondcdemo.obsrv.ai/superset/dashboard/ondc-dashboard/
Credentials - ondcdemo/ondc$123

[bharath-ganapathy]

Is there a network dashboard for ONEST? even a mock/wireframe would be a good addition to this thread.

the link to the ONDC Demo Dashboard seems to be broken, i got a timed_out error.