Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edge-agent scouting design spec #423

Open
wants to merge 1 commit into
base: testing/v4
Choose a base branch
from
Open

Edge-agent scouting design spec #423

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/edge-scout.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Edge Agent Scout Mode (Draft)

## Objective
Currently, the Edge Agent receives a configuration containing manually set addresses for data sources, which are defined in the ACS config service and then sent to the Edge Agent to retrieve data.

The challenge with this approach is that the addresses must be known in advance, typically provided by manufacturers per device. However, these addresses can be incorrect or vary between devices.

The goal of this new feature is to develop an MVP to explore whether this manual process can be reversed. Instead of predefining addresses, the Edge Agent would run in scout mode, actively requesting devices to provide their actual list of available addresses. This functionality would need to be implemented for each supported communication protocol, such as OPC UA, MQTT, and others.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going into the ACS docs as the documentation of the scouting feature, so it should be written as though the feature has been implemented and merged into ACS. 'The scouting feature allows this manual process to be reversed' etc.

Once the Edge Agent receives the list of data sources from the devices, it should store this information in the ACS config service.


## Architecture
This feature will extend the existing Edge Agent source code. The ACS Config Service configuration will be updated to specify whether the Edge Agent should run in Scout mode and may include additional connection-specific configurations if needed.

If Scout mode is enabled, the Edge Agent will trigger the scouting method for each supported connection type (OPC UA, MQTT, etc.). Once the list of addresses is received, the Edge Agent will send it to the ACS Config Service to be stored within a new object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The discovered addresses want to be stored using a new Application called something like Edge scout results against the UUID of the Connection. The Connections should be pre-registered in the ConfigDB by the time you get to this point, and their UUIDs should be in the Edge Agent config so you know what they are, but we haven't got there yet:

  • Migrate Connections into the ConfigDB #406 migrates existing connection information out of the Manager database and into the ConfigDB. This handles upgrading existing ACS installations.
  • However the existing (v3) Manager hasn't been changed to keep these up to date when new connections are created, because we're getting rid of it.
  • The Manager should also be updated to include the UUID for each Connection in the Edge Agent config, but it also doesn't do that yet, for the same reason.
  • The new version of the Manager will hopefully do both these things but won't be usable for a while yet.

When you get to the point of implementing this we will need to work out how to move forward. I would allocate the Application UUID now and document it here.


![High Level Diagram](https://github.com/AMRC-FactoryPlus/amrc-connectivity-stack/blob/za/edge_agent_scout_mode/docs/assets/edge-agent-scout/acs-edge-scout-high-level-diagram.png?raw=true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image URL should be relative ./assets/.... Otherwise it breaks when the PR is merged.


## Implementation
### Translator Class (Changes for Scouting Feature)
The scouting feature is embedded into the Edge Agent's Translator class.
- The Translator class checks if the connection configuration includes a scout entry.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no spec anywhere for this scout property. I know that it's a property of one of the deviceConnections in the Edge Agent config ConfigDB entry, but someone new to ACS reading this document won't know that.

The format of the value of the scout property also needs documenting; there should be a general comment 'the value of this property depends on the driver' and then driver-specific documentation below. If there is a Scout-specific section and a driver-specific section then the Scout parts can be documented in detail here.

- If the current connection is configured to run in scout mode, the Translator creates an instance of the Scout class for this connection, passing the DeviceConnection instance and scoutConfig as arguments to the constructor:
`constructor(deviceConnection: DeviceConnection, scoutConfig: any)`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this scoutConfig the whole of the value of the scout property or only part of it?

- Upon creating the Scout instance, the Translator subscribes to the **scoutComplete** event, which returns a list of discovered addresses. The Translator then sends this list back to the ACS Config Service.
- Finally, the Translator calls the Scout instance's `performScouting()` method to perform scouting.

### Scout Class (New Component)
- The Scout class receives the DeviceConnection instance and scoutConfig.
- The Scout class understands the expected configuration format for each type of DeviceConnection and converts it accordingly (e.g., MQTT, OPC UA, etc.).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Scout class should not be performing validation of the driver-specific part of the scout config. This will be impossible with external drivers. If validation of the scout config is needed this should be another (async) method on the Device.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to note: it's important the validation method is async, as for an external driver the Edge Agent code will need to communicate with the driver to perform the validation.

- The `performScouting()` method invokes the relevant method of the DeviceConnection to perform scouting.
- Once the DeviceConnection completes scouting, `performScouting()` receives a list of discovered addresses.
- Finally, the method triggers the **scoutComplete** event with the list of discovered addresses.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not important for MVP, but eventually I think it will be helpful to re-run the scouting every N minutes, or to allow the Device to support continuous scouting with a stream of scoutComplete events. So:

  • It might be better to call the event scoutResults.
  • It might be easier to have the Device emit the event, and the Scout push the results up to the ConfigDB.
  • You definitely want a the scoutConfig to be in two pieces, one for the Device and one for the Scout class itself.


### DeviceConnection (Changes for Scouting Feature)
- A new method is added to the abstract DeviceConnection class:
`public async scoutAddresses(scoutDetails: ScoutDetails): Promise<string[]>`
- This method must be implemented in its subclasses (MQTTConnection, OPCUAConnection), as the process for retrieving available addresses varies based on the protocol.

### MQTTConnection (Changes for Scouting Feature)
MQTTConnection implements the `public async scoutAddresses(scoutDetails: ScoutDetails): Promise<string[]>` of its superclass DeviceConnection.
- The `scoutAddresses` method for MQTTConnection retrieves the topic and duration from its configuration.
These parameters define:
- Topic: The topic to listen to (e.g., #).
- Duration: The length of time to listen (e.g., 5 minutes).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the information that is needed in the spec for the scoutConfig for MQTT, but it needs to be precise. For example: is the property called Topic or topic? A JSON schema for the MQTT-specific configuration might be helpful, or just a table Property/Type/Meaning as elsewhere in the ACS docs.


- The MQTTConnection creates a dedicated MQTT client for scouting, which:
- Subscribes to the specified topic.
- Listens for the given duration.
- Collects all topics received during this period.
- Returns the list of discovered topics to the Scout class.
- Shuts down the MQTT client after scouting is complete.

### OPCUAConnection (Changes for Scouting Feature)
OPCUAConnection implements the `public async scoutAddresses(scoutDetails: ScoutDetails): Promise<string[]>` of its superclass DeviceConnection.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does OPCUA need any configuration for scouting or is there no choice about what to do here?

- The OPCUAConnection creates an OPC UA client and a session that recursively searches for all nodes within the provided OPCUA server starting from the RootNode.
- Certain Node Classes can contain child nodes, so their children are also checked. These classes include:
- Object (1)
- ObjectType (8)
- ReferenceType (32)

- If a node belongs to a leaf node class, it is not further traversed. Leaf node classes include:
- Variable (2)
- VariableType (16)
- Method (4)
- DataType (64)
- View (128)

- Nodes of the Variable class (2) are stored in a list of unique values and returned to the Scout class.