Skip to content

Datum Input Endpoint

Jump to bottom Edit New page
Matt Magoffin edited this page Jun 3, 2024 · 15 revisions

SolarNetwork provides a fully-managed Datum Input Endpoint (DIN) HTTP integration layer that allows arbitrary real-time measurements to be ingested as a SolarNetwork datum stream. This can be used to integrate measurements posted directly from devices without using SolarNode to collect the measurements.

The system works like this:

  • You create a Transform configuration that specifies a Transform Service supported by SolarNetwork. A Transform Service is responsible for taking some input measurement data and converting it to native SolarNetwork datum.
  • You create an Endpoint configuration that specifies attributes like the node ID and source ID to apply to the converted datum, and the Transform configuration to use. The Endpoint will define a unique Endpoint ID that becomes part of the HTTP URL used to post the measurement data to SolarNetwork.
  • You create a Credential configuration that defines a username and password that must be included when posting the measurement data to SolarNetwork.
  • You create an Endpoint Authorization configuration that associates the Credential with the Endpoint.

The XSLT Transform Service is one example of a Transform Service; it allows you to configure an XSLT stylesheet that can convert arbitrary XML or JSON data into a SolarNetwork datum stream.

Posting data

Once an Endpoint is configured with at least one Endpoint Authorization, measurement data can be sent to the endpoint via HTTP POST, to a URL in this form:

POST https://din.solarnetwork.net/api/v1/datum/endpoint/{endpointId}
endpointId The endpoint ID of the Endpoint to post to.

☝️ Note the {endpointId} portion of the URL must be changed to the ID of an actual Endpoint.

HTTP basic authorization must be included with an Autorization: Basic HTTP header, using any valid credentials associated with the endpoint.

An example HTTP POST request to an Endpoint with ID d4f3d589-31fd-4005-b780-b0593f943910 looks like this:

POST /api/v1/datum/endpoint/d4f3d589-31fd-4005-b780-b0593f943910 HTTP/1.1
Authorization: Basic Zm9vYmFyOnNlY3JldA==
Accept: application/json
Content-Type: application/json
Host: din.solarnetwork.net:443
Content-Length: 73

{"ts":"2024-02-22T12:00:00Z","props":{"foo":123,"bim":234,"msg":"Hello"}}

Request HTTP headers

The following HTTP headers are utilized:

Header Required Description
Accept Yes The desired response body content type.
Content-Type Yes The request body content type.
Accept-Encoding No Can be set to gzip to receive a compressed response body.
Content-Encoding No Can be set to gzip to provide a compressed request body.

Content types

The request Accept and Content-Type HTTP headers must be provided. As long as the configured request transform supports the provided Content-Type value and the response transform supports the provided Accept value, the response Content-Type will be set to the request Accept value.

Request query parameters

URL query parameters may be provided, and they will be passed to the Transform configured on the endpoint. In addition, the nodeId and sourceId parameters can be used to provide the node and source ID values to use for the generated datum stream. This can be useful so a single Endpoint can process input data from many identical devices, each with their own node or source ID values.

For example, a URL path that specifies node 123 and source meter looks like this (the https://din.solarnetwork.net prefix is omitted for brevity):

/api/v1/datum/endpoint/d4f3d589-31fd-4005-b780-b0593f943910?nodeId=123&sourceId=meter1

Request body compression

The request body can be compressed using the gzip compression scheme. Include a Content-Encoding HTTP header, like this:

Content-Encoding: gzip

Data length limit

The measurement data must be no more than 8 KB in length, otherwise an error will be returned. This limit applies to the uncompressed data, if HTTP gzip encoding is used.

Posting data response

The response will be a list of datum ID objects of all successfully converted datum. Each object has the following properties:

Property Type Description
kind String Will be Node.
objectId Number The node ID of the datum.
sourceId String The source ID of the datum.
timestamp String The timestamp of the datum. 
{
  "success": true,
  "data": [
    {
      "kind": "Node",
      "objectId": 123,
      "sourceId": "meter/1",
      "timestamp": "2024-02-22 12:00:00Z"
    }
  ]
}

Transform Services

A Transform Service is responsible for converting measurement data into native SolarNetwork datum.

XSLT Transform Service

The XSLT Transform Service uses XSLT (that you provide) to transform XML or JSON measurement data into native SolarNetwork datum JSON. It can thus handle a wide variety of input data forms and is a good choice when no other more specific Transform Service is available.

The native SolarNetwork datum JSON object that the XSLT must produce looks like this:

{
  "created": "2024-02-22T12:00:00Z",
  "nodeId": 123,
  "sourceId": "meter/1",
  "samples": {
    "i": {
      "watts": 123,
      "frequency": 50.1,
      "voltage": 244
    },
    "a": {
      "wattHours": 987654321
    },
    "s": {
      "mode": "auto"
    },
    "t": [
      "online"
    ]
  }
}

☝️ Note that the nodeId and sourceId values are not needed in the JSON output if the Endpoint defines those values.

The samples object properties can instead be included directly, like this:

{
  "created": "2024-02-22T12:00:00Z",
  "nodeId": 123,
  "sourceId": "meter/1",
  "i": {
    "watts": 123,
    "frequency": 50.1,
    "voltage": 244
  },
  "a": {
    "wattHours": 987654321
  },
  "s": {
    "mode": "auto"
  },
  "t": [
    "online"
  ]
}

A JSON array of datum is also supported, for example like this:

[
  {
    "created": "2024-02-22T12:00:00Z",
    "nodeId": 123,
    "sourceId": "meter/1",
    "samples": {
      "i": {
        "watts": 123,
      },
      "a": {
        "wattHours": 987654321
      }
    }
  },
  {
    "created": "2024-02-22T12:01:00Z",
    "nodeId": 123,
    "sourceId": "meter/1",
    "samples": {
      "i": {
        "watts": 99,
      },
      "a": {
        "wattHours": 987654387
      }
    }
  }
]

XSLT up to version 3 is supported. The XSLT stylesheet you provide will not be allowed to resolve external entities or resources, so things like <xsl:import> are not supported.

SolarDIN XSLT transform data flow diagram

Supported content types

The following request Content-Type HTTP header values are supported:

  • application/json
  • application/*+json
  • application/*+xml
  • text/xml

The request Accept HTTP header value will be used as-is for the response Content-Type HTTP header.

XSLT Transform Service properties

The XSLT Transform Service has a Service ID of net.solarnetwork.central.din.XsltTransformService and accepts the following service properties:

Property Description
xslt The XSLT stylesheet that produces datum JSON from the input data. If the input data is JSON then an XSLT input parameter input-json will be provided with the JSON string value. You can use the XSLT 3 function json-to-xml($input-json) to transform that into XML, and then use normal XSLT templates to perform the conversion to native SolarNetwork datum JSON.
cache-seconds An optional templates cache period, in seconds. Typically you do not need to specify this.

XSLT Transform input parameters

The transform will be provided with the following input parameters, which you can access with XSLT <xsl:param> declarations:

Parameter Type Description
input-json string If the input data is JSON, this parameter will be set to the JSON value (and the XSLT will be provided an empty XML document).
user-id integer The SolarNetwork user ID that owns the Transform configuration.
endpoint-id string The ID of the Endpoint configuration being used.
transform-id integer The ID of the Transform configuration being used.
preview boolean Will be true if the transform is for preview purposes.

Datum Input Endpoint entities

Within SolarNetwork, the following entities are managed via the SolarUser Datum Input Endpoint API:

SolarDIN entity relationship diagram

Credential entity

Username and password details that can be associated with Endpoint Authorizations.

See the Credential API for details.

Transform entity

Transform Service configuration, which is specific to the Transform Service configured.

See the Transform API for details.

Endpoint entity

Defines a unique data input endpoint, with the Transform to use and node and source ID values to assign to the converted datum.

See the Endpoint API for details.

Endpoint Authorization entity

Associates a username/password Credential to an Endpoint, allowing those credentials to be used to post data to the endpoint. Any number of Endpoint Authorization entities can be associated with a given Endpoint, and a given Credential can be associated with any number of Endpoint Authorizations. Or put another way, you can use the same credentials across different endpoints, and configure multiple credentials on a single endpoint.

See the Endpoint Authorization API for details.

Add a custom footer
Clone this wiki locally