diff --git a/docs/_assets/action-states.png b/docs/_assets/action-states.png new file mode 100644 index 0000000..90f094a Binary files /dev/null and b/docs/_assets/action-states.png differ diff --git a/docs/_assets/aggregate-states.png b/docs/_assets/aggregate-states.png new file mode 100644 index 0000000..038d376 Binary files /dev/null and b/docs/_assets/aggregate-states.png differ diff --git a/docs/_assets/device-inventory.png b/docs/_assets/device-inventory.png new file mode 100644 index 0000000..2a6247b Binary files /dev/null and b/docs/_assets/device-inventory.png differ diff --git a/docs/current-state-specification.md b/docs/current-state-specification.md new file mode 100755 index 0000000..7c744fd --- /dev/null +++ b/docs/current-state-specification.md @@ -0,0 +1,174 @@ +### Current State Representation + +The cloud backend (or any OTA update system) needs to keep track of the current state of the device. This current state information is relevant as input for the context pipeline for compiling future desired states and acts as an indicator of problems in applying a given desired state, if the current state differs from the desired state, which may not be reached. + +For representing the current state of the device, the approach of a holistic inventory of hardware and software nodes will be applied. This model is using a graph structure consisting of three element types - hardware nodes, software nodes, associations. + +### Current State Data Model + +The following table describes all supported properties and sections of the Current State specification: + +| Property | Type | Description | +| - | - | - | +| **General properties** | | | +| hardwareNodes | JSON array | Inventory for a list of hardware nodes | +| softwareNodes | JSON array | Inventory for a list of software nodes | +| associations | JSON array | List of mappings between the inventory nodes. No semantics on model level, just a link between two nodes - either software, or hardware nodes. For Update Manager semantics, see [Device Inventory Graph Representation](#device-inventory-graph-representation) below. | +| **Hardware node properties** | | | +| id | string | Identifier of the hardware node | +| version | string | Version of the hardware node | +| name | string | Name of a hardware node | +| parameters | JSON array | List of key/value parameters for a hardware node. The parameters are solution and domain-specific. Detailed documentation for the supported key-value parameters of a hardware node are to be provided additionally. | +| addressable | boolean | Enables hardware node addressability | +| **Software node properties** | | | +| id | string | Identifier of the software node | +| version | string | Version of the software node | +| name | string | Name of the software node | +| parameters | JSON array | List of key/value parameters for a software node. The parameters are solution and domain-specific. Detailed documentation for the supported key-value parameters of a software node are to be provided additionally. | +| type | string | Type of the software node. The supported types are listed [below](#supported-software-types) | +| **Parameter properties** | | | +| key | string | Key of the parameter | +| value | string | Value of the parameter | +| **Association parameter** | | | +| sourceId | string | Identifier of the source node of the association | +| targetId | string | Identifier of the target node of the association | + +### Supported software types + +The list of the supported software types : + +| Type | Description | +| - | - | +| IMAGE | Represents an image software type | +| RAW | Represents a raw bytes software type | +| DATA | Represents a data software type | +| APPLICATION | Represents an application software type | +| CONTAINER | Represents a container software type | + +### Device Inventory Graph Representation + +The diagram below represents the Device Inventory graph and the links between the software and hardware nodes. + +The software nodes are organized in the graph at different levels. +- At the root level stands the main Update Manager software node (type APPLICATION), which represents the Update Manager component. +- At the second level are placed the software nodes for each domain update agent (type APPLICATION), which are linked with associations to the Update Manager node. These software nodes should also have a parameter with key `domain` to specify the domain they are responsible for. +- At the last level in the hierarchy are placed the domain-specific software nodes, representing the domain components. These software nodes can be modeled the in a tree-based structure if the internal domain-specific representation is more complex. This is domain-specific, extra documentation should come from the respective domain update agent, e.g. Eclipse Kanto containers update agent that is part of the Eclipse Kanto Container Management component. + +The hardware nodes do not follow any strict hierarchy and can be linked to any hardware or software node. Any cycles between the nodes are not allowed and prevented. + +![Device inventory](./_assets/device-inventory.png) + +### Current State Data Model Example + +The following data structure is a holistic example view of a device current state: +```json +{ + "hardwareNodes": [ + { + "id": "cOffee", + "version": "rev2", + "name": "OWASYS box", + "parameters": [ + { + "key": "cpu-arch", + "value": "armv7" + } + ] + } + ], + "softwareNodes": [ + { + "id": "update-manager", + "version": "1.0.0", + "name": "Update Manager", + "type": "APPLICATION" + }, + { + "id": "containers-update-agent", + "version": "1.0", + "name": "Containers Update Agent", + "type": "APPLICATION", + "parameters": [ + { + "key": "domain", + "value": "containers" + }, + { + "key": "container_registry", + "value": "ghcr.io" + } + ] + }, + { + "id": "containers:hello-world", + "version": "latest", + "type": "CONTAINER", + "parameters": [ + { + "key": "image", + "value": "docker.io/library/hello-world:latest" + }, + { + "key": "status", + "value": "Running" + } + ] + }, + { + "id": "containers:influxdb", + "version": "2.5", + "type": "CONTAINER", + "parameters": [ + { + "key": "image", + "value": "docker.io/library/influxdb:2.5" + }, + { + "key": "status", + "value": "Running" + } + ] + }, + { + "id": "self-update-agent", + "version": "0.2.0", + "name": "Self Update Agent", + "type": "APPLICATION", + "parameters": [ + { + "key": "domain", + "value": "self-update" + } + ] + }, + { + "id": "self-update:leda-deviceimage", + "version": "1.0.0", + "name": "Official Leda Device Image", + "type": "IMAGE" + } + ], + "associations": [ + { + "sourceId": "update-manager", + "targetId": "containers-update-agent" + }, + { + "sourceId": "update-manager", + "targetId": "self-update-agent" + }, + { + "sourceId": "containers-update-agent", + "targetId": "containers:hello-world" + }, + { + "sourceId": "containers-update-agent", + "targetId": "containers:influxdb" + }, + { + "sourceId": "self-update-agent", + "targetId": "self-update:leda-deviceimage" + } + ] +} +``` diff --git a/docs/desired-state-feedback-specification.md b/docs/desired-state-feedback-specification.md new file mode 100644 index 0000000..4c74a8f --- /dev/null +++ b/docs/desired-state-feedback-specification.md @@ -0,0 +1,136 @@ +### Desired State Feedback Representation +The cloud backend (or OTA update system) needs to keep track of the update progress on both individual level as well as on campaign (or fleet) level. To achieve this purpose, the Update Manager introduces the Desired State Feedback as a common, structured and straightforward way for notifying the backend about the progress and status of the update. + +The feedback is comprised by overall information for the current status of the ongoing update process, information for the status for each of the identified components and optional unstructured message. This message is meant to support projects with the need to transfer custom data such as a human readable progress information or internally used status codes. +Optionally, the feedback can be defined at baseline level by specifying additional information for the baseline the feedback is associated with. If not present, the feedback is considered to be related at domain level. + +### Desired State Feedback Data Model + +The following table describes all supported properties and sections of the Desired State specification: + +| Property | Type | Description | +| - | - | - | +| **General properties** | | | +| status | string | [Status of the Desired State](#supported-desired-state-feedback-statuses) | +| baseline | string | Title of the baseline, the Desired State Feedback is associated with. Optional | +| message | string | Info message for the Desired State Feedback | +| actions | JSON array | Set of Desired State Feedback actions | +| **Action properties** | | | +| component | JSON object | The component, which is linked with the action | +| status | string | [Status the of action](#supported-desired-state-feedback-action-statuses) | +| progress | int | Progress of the action in percentage | +| message | string | Info message for the action | +| **Config properties** | | | +| key | string | Key of the configuration property | +| value | string | Value of the configuration property | +| **Component properties** | | | +| id | string | Identifier of the component | +| version | string | Version of the component | +| config | JSON object | Set of component-specific configuration properties as key/value pairs | + +### Supported Desired State Feedback statuses + +The list of the supported feedback statuses : + +| Status | Description | +| - | - | +| IDENTIFYING | Denotes that action identification process has started | +| IDENTIFIED | Denotes that required actions are identified | +| IDENTIFICATION_FAILED | Denotes some or all of the required actions failed to be indentified | +| RUNNING | Denotes that identified actions are currently executing | +| DOWNLOADED_COMPLETED | Denotes that identified actions are downloaded successfully | +| DOWNLOADED_FAILED | Denotes that not all of the identified actions are downloaded successfully | +| UPDATE_COMPLETED | Denotes that identified actions are updated successfully | +| UPDATE_FAILED | Denotes that not all identified actions are updated successfully | +| ACTIVATION_COMPLETED | Denotes that identified actions are activated successfully | +| ACTIVATION_FAILED | Denotes that not all identified actions are activated successfully | +| COMPLETED | Denotes that identified actions completed successfully | +| INCOMPLETE | Denotes that not all of the identified actions completed successfully | +| INCOMPLETE_INCONSISTENT | Denotes that not all of the identified actions completed successfully, leaving the state inconsistent | +| SUPERSEDED | Denotes that the identified actions are no longer valid because new desired state was requested | + +### Supported Desired State Feedback Action statuses + +The list of the supported action statuses : + +| Status | Description | +| - | - | +| IDENTIFIED | Denotes that action is identified, initial status for each action | +| DOWNLOADING | Denotes an artifact is currently being downloaded | +| DOWNLOADED_SUCCESS | Denotes an artifact has been downloaded successfully | +| DOWNLOADED_FAILURE | Denotes an artifact an artifact download has failed | +| UPDATING | Denotes a component is currently being installed or modified | +| UPDATE_SUCCESS | Denotes a component has been installed or modified successfully | +| UPDATE_FAILURE | Denotes a component could not be installed or modified | +| ACTIVATING | Denotes a component is currently being activated | +| ACTIVATION_SUCCESS | Denotes a component has been activated successfully | +| ACTIVATION_FAILURE | Denotes a component could not be activated | +| REMOVING | Denotes a component is currently being removed | +| REMOVAL_SUCCESS | Denotes a component has been removed successfully | +| REMOVAL_FAILURE | Denotes a component could not be removed | + +### Desired State Feedback Data Model Example + +This is a full example of a desired state feedback message for a device that is currently applying a desired state. In this example, three actions have been identified of which two are still running: +```json +{ + "status": "RUNNING", + "message": "Applying desired state across 3 agents. Estimated time: about 5 minutes.", + "actions": [ + { + "component": { + "id": "containers:xyz", + "version": "1" + }, + "status": "UPDATE_SUCCESS", + "message": "Container is healthy. Startup time: 7 seconds" + }, + { + "component": { + "id": "custom-domain:app-1", + "version": "4.3" + }, + "status": "DOWNLOADING", + "progress": 79, + "message": "Downloading 53.8 MiB" + }, + { + "component": { + "id": "custom-domain:app-2", + "version": "342.444.195" + }, + "status": "UPDATING", + "progress": 79, + "message": "Writing firmware" + } + ] +} +``` + +### Desired State Feedback Action Representation + +A desired state application can lead to a multitude of actions that need to be identified and performed by the device. An action can either represent the installation or the uninstallation / removal of a component. + +### Desired State Feedback Action Data Model + +The data model for each individual action is specified as follows. It references the affected components version and id as per the desired state as well as the status and an optional message. +``` +component: + id: identifier of this action's component + version: version of this action's component +status: IDENTIFIED|DOWNLOADING|DOWNLOAD_FAILURE|DOWNLOAD_SUCCESS|UPDATING|UPDATE_FAILURE|UPDATE_SUCCESS|ACTIVATING|ACTIVATION_FAILURE|ACTIVATION_SUCCESS|REMOVING|REMOVAL_FAILURE|REMOVAL_SUCCESS +progress: Optional progress information - percentage (Integer 0..100) +message: Optional additional info about this action, e.g. internal status code, error message, ... +``` + +### Desired State Feedback Action State + +The following diagram depicts the action state transitions: + +![Action states](./_assets/action-states.png) + +### Device Level Aggregated State + +The overall device level state of the desired state application depends on the states of each individual action that has been identified. Only if all of them have completed successfully, the device has reached its desired state. The following diagram depicts the state transitions: + +![Aggregated states](./_assets/aggregate-states.png) diff --git a/docs/desired-state-specification.md b/docs/desired-state-specification.md new file mode 100644 index 0000000..56c709e --- /dev/null +++ b/docs/desired-state-specification.md @@ -0,0 +1,136 @@ +### Desired State Representation +A technology agnostic way of representing the target software state of the device is required to enable remote management of device software and configuration. The device software state model enables flexible representation of complex software topologies of a device as well as the related configuration. It also enables users and application developers to develop and deploy composite applications seamlessly based on device context changes. + +### Desired State Data Model + +The following table describes all supported properties and sections of the Desired State specification: + +| Property | Type | Description | +| - | - | - | +| **General properties** | | | | +| baselines | JSON array | List of Desired State baselines | +| domains | JSON array | List of Desired State specifications for the domain agents | +| **Baseline properties** | | | +| title | string | Title of the baseline | +| description | string | Description of the baseline. A baseline is supposed to hold dependent components that are to be updated together. | +| preconditions | string | List of comma-separated precondition expressions | +| components | JSON array | Components of the baseline, each entry is prefixed with the domain name so that cross-domain dependecies can be handled too. | +| **Domain properties** | | | +| id | string | Identifier of the domain agent | +| config | JSON object | Set of domain-specific configuration properties as key/value pairs | +| components | JSON object | Set of components for the domain | +| **Config properties** | | | +| key | string | Key of the configuration property | +| value | string | Value of the configuration property | +| **Component properties** | | | +| id | string | Identifier of the component | +| version | string | Version of the component | +| config | JSON object | Set of component-specific runtime configuration properties as key/value pairs | + +### Desired State Data Model Example + +The following data structure is a holistic example of a device desired state: +```json +{ + "baselines": [ + { + "title": "baseline-1", + "description": "Bugfix because of problem 1", + "preconditions": "AND(device.mode=MAINTENANCE,OR(device.battery>75,device.pluggedIn=true))", + "components": [ + "custom-domain:app-1", + "custom-domain:app-2" + ] + }, + { + "title": "composite-app123", + "components": [ + "containers:xyz", + "custom-domain:app-3", + "custom-domain:app-4" + ] + } + ], + "domains": [ + { + "id": "containers", + "components": [ + { + "id": "xyz", + "version": "1.2.3", + "config": [ + { + "key": "image", + "value": "container-registry.io/xyz:1.2.3" + } + ] + }, + { + "id": "abc", + "version": "4.5.6", + "config": [ + { + "key": "image", + "value": "container-registry.io/abc:4.5.6" + } + ] + } + ] + }, + { + "id": "custom-domain", + "components": [ + { + "id": "app-1", + "version": "1.0" + }, + { + "id": "app-2", + "version": "4.3" + }, + { + "id": "app-3", + "version": "342.444.195", + "config": [ + { + "key": "some.setting", + "value": "abcd" + } + ] + }, + { + "id": "app-4", + "version": "568.484.195" + } + ] + }, + { + "id": "self-update", + "config": [ + { + "key": "rebootRequired", + "value": "true" + } + ], + "components": [ + { + "id": "os-image", + "version": "https://example.com/image.tar.gz" + } + ] + } + ] +} +``` + +### Configuration State Representation +Besides software components, the desired state representation needs to also support configuration state. This is used to provide runtime configuration to the components such as environment variables or secrets. In the state representation model, each software component has a set of configuration attached to it in a 1:1 relationship. + +### Data Model and Domains +The purpose of this model is to describe the required data structure in a technology agnostic way. +The desired software state covers multiple domains of edge device, which can be extended as additional domains are exploited through newly developed update agents. +The device software state model is capable of supporting any domains. This is achieved by abstracting the domain specific installation technology via corresponding update agents. + +### Dependencies between component updates +When transmitting a desired state to a device, the device needs to be informed about dependencies between the contained components e.g. a composite app can comprise of updates to components in different domains. +These requirements make it necessary to transmit such implicit dependencies from backend to the device. To achieve this flexibility, the dependencies are modeled by a root-level object `baselines` which is used to describe dependencies between elements from the domain's respective components sections. Each baseline represent a logical unit, which is comprised of set of components across multiple domains.