Add converter for dashboard JSON in "export format" representation

[amotl]

Hi there,

@jeremybz / @jeremybanzhaf asked at ¹:

My goal is to use the API to get the JSON representation of a dashboard, including __inputs and named datasources.

@jangaraj answered:

HTTP API for dashboard export for external use doesn’t exist. This export feature is implemented in the UI (frontend). I would say the only option is to use standard dashboard API and make that output transformed into “dashboard format for external use”. Unfortunately, that format is not documented, so you have to use reverse engineering.

Torkel Ödegaard said at ²:

This is not possible, the data is captured by the frontend so you would have to replicate what all the panel & data sources do (issue metric queries etc).

In order to implement this transformation, I would be happy to provide a place within this package, if possible. Would that make any sense / be helpful in any way?

The implementation should adhere to the specifications defined by the corresponding TypeScript code.

• DashboardModel::getSaveModelClone()
• DashboardModel::getPanelSaveModels()
• DashboardExporter.ts
• DashboardExporter.test.ts

With kind regards,
Andreas.

/cc @jangaraj

Footnotes

1. https://community.grafana.com/t/export-dashboard-for-external-use-via-http-api/50716 ↩

2. https://community.grafana.com/t/snapshot-api-how-to-get-the-data/2424 ↩

[jangaraj]

I agree. That DashboardExporter.ts needs a corresponding Python implementation.

[amotl]

Hi again,

I made a start with model/dashboard.py ¹ within the collab/dashboard-export branch ². The main body of DashboardExporter.makeExportable will still have to be implemented, but the framework is there ³.

With kind regards,
Andreas.

Footnotes

1. https://github.com/panodata/grafana-client/blob/collab/dashboard-export/grafana_client/model/dashboard.py ↩

2. https://github.com/panodata/grafana-client/compare/collab/dashboard-export ↩

3. Supported by a preliminary but working command line invocation like python -m grafana_client.model.dashboard play.grafana.org 000000012 | jq. ↩

[amotl]

The main body [...] will still have to be implemented [...]

It might have happened that @peekjef72 implemented the corresponding nitty-gritty details with their grafana-snapshots-tool already?

[peekjef72]

Hi,
grafana-snapshot-tool as mentionned in its documentation, can import, export, and generate what is called in GRAFANA UI "snapshots": they are dashboards with corresponding datas incorporated.
My opinion is that snapshots, can't be part of the API because they are composed of calls to others parts of the api:

• exporting a dashboard
• collection datasources names
• parsing the datasources used by the dashboard
• query datasources from grafana
• incorporating results into the dashboard
• then generating a file or importing it to a grafana server.

A snapshot is a picture of a dashboard too, as a consequence it has parameters:

• a time range (timefrom, time to)
• value for each variables used by the dashboard.

Grafana-snapshot-tool is my implementation of these functionnalities.
It uses the python api (panadata/grafana-client), to perform all above actions.
My contribution to this repository, concerns datasources, particullary the ability to query them, as required and done by Grafana UI (reverse engineering on API call with browser dev tool).
Hope my opinion can help.
Anyway feel free to use the code or to incorpore it into contribs, if you think it should help :)

[amotl]

Dear Jean-Francois,

apologies for the late reply. I am very happy that we finally got closer in touch at grafana-toolbox/grafana-snapshots-tool#2 (comment) ff. If I can find some time to work on another iteration of grafana-client, I will also be happy to share further thoughts about our topics of shared interest.

In the meanwhile, I will be looking forward to all contributions and improvements coming from your pen which may be needed as we go. Thank you very much already for making a start with main...peekjef72:grafana-client:main.

With kind regards,
Andreas.

[digitronik]

@amotl I don't know current state of this but with some reverse engineering,
I tried to solve my externally shareable dashboard export... I think grafana client should provide some more generic implementation for shareable dashboards.

class Grafana:
    """A class to interact with Grafana's API for managing dashboards."""

    def __init__(self, host: str, api_key: str = None):
        """Initializes the Grafana instance.

        Args:
            host (str): The base URL of the Grafana instance.
            api_key (str, optional): The API key for authenticating requests.
        """
        self.host = host
        self.api_key = api_key

    @property
    def headers(self):
        """Returns the headers for API requests."""
        return {"Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json"}

    @cached_property
    def datasources(self) -> dict | None:
        """Fetches and caches the available datasources from Grafana."""
        datasources = {}
        url = urljoin(self.host, "/api/datasources")
        response = requests.get(url, headers=self.headers)

        if response.status_code == 200:
            for ds in response.json():
                datasources[ds["uid"]] = {
                    "name": ds["name"].replace("-", "_").upper(),
                    "label": ds["name"],
                    "description": "",
                    "type": "datasource",
                    "pluginId": ds["type"],
                    "pluginName": ds["typeName"],
                }
            return datasources
        else:
            print(f"Failed to fetch datasources: {response.status_code} - {response.text}")
            return None

    def _external_sharable(self, dashboard: dict) -> dict:
        """Prepares a dashboard for external sharing by updating datasource references.

        Args:
            dashboard (dict): The dashboard JSON data.

        Returns:
            dict: The updated dashboard JSON data.
        """
        datasources = self.datasources
        applicable_ds = []

        def _search_replace_datasource(obj):
            if isinstance(obj, dict):
                for key, value in obj.items():
                    if key == "annotations":
                        continue
                    # Replace datasource UID with variable.
                    if key == "datasource" and isinstance(value, dict):
                        if uid := value.get("uid"):
                            if uid in datasources:
                                applicable_ds.append(uid)
                                ds_name = datasources[uid]["name"]
                                value["uid"] = f"${{{ds_name}}}"
                    # Replace sitreps API's UIL with Variable.
                    elif key == "url" and re.search(SITREPS_API_REGEX, value):
                        obj[key] = re.sub(SITREPS_API_REGEX, "${SITREPS_API}", value)
                    else:
                        _search_replace_datasource(value)
            elif isinstance(obj, list):
                for item in obj:
                    _search_replace_datasource(item)

        _search_replace_datasource(dashboard)
        inputs = [datasources.get(ds_uid) for ds_uid in set(applicable_ds)]

        # If sitreps apis used by dashobard then include input asking hostname for sitreps apis.
        if "APIS" in [ds["name"] for ds in inputs]:
            inputs.append(
                {
                    "name": "SITREPS_API",
                    "label": "sitreps_api",
                    "description": "Sitreps API endpoint.",
                    "type": "constant",
                }
            )

        dashboard["__inputs"] = inputs
        return dashboard

[amotl]

Dear @digitronik,

your contribution is well received. Currently, I am a bit short on time, but if you see a chance to slot your code into (on top of) GH-23 in one way or another, a corresponding patch will be even more welcome.

With kind regards,
Andreas.