Support custom fragments in alarm creation #1699

PradeepKiruvale · 2023-01-30T07:02:13Z

Signed-off-by: Pradeep Kumar K J pradeepkumar.kj@softwareag.com

Proposed changes

Create an alarm with custom fragments
Supports for both thin-edge devices as well as for the child devices.
For example, if an alarm message is published with a custom fragment as below a child device it must create an alarm with the custom message.

tedge mqtt pub tedge/alarms/critical/myCustomAlarmType/child_device_example '{ "text": "Some test alarm", "someOtherCustomFragment": {"nested":{"value": "extra info"}} }'

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
Documentation Update (if none of the other choices apply)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#1682

Checklist

I have read the CONTRIBUTING doc
I have signed the CLA (in all commits with git commit -s)
I ran cargo fmt as mentioned in CODING_GUIDELINES
I used cargo clippy as mentioned in CODING_GUIDELINES
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

Further comments

reubenmiller · 2023-01-30T10:46:31Z

@PradeepKiruvale Look ok from a non-rust pov. I will let @albinsuresh (or some other rust enabled person) to formally approve to prevent it from being merged prematurely.

crates/core/tedge_mapper/src/c8y/alarm_converter.rs

crates/core/tedge_api/src/alarm.rs

didier-wenzek · 2023-01-30T12:13:00Z

crates/core/tedge_api/src/alarm.rs

 pub struct ThinEdgeAlarmData {
    pub text: Option<String>,

    #[serde(default)]
    #[serde(with = "time::serde::rfc3339::option")]
    pub time: Option<Timestamp>,
+
+    #[serde(flatten)]
+    pub extras: HashMap<String, Value>,


Why is this named extras? We use the term fragment in the documentation.

I agree, we can rename it as fragments. But it will also contain the child device info as part of the message. Can this child device info also be called a fragment? I am not sure. So, I kept it in line with the events.

If the child-device-id is stored as an entry in this extras map, what's the content of the source ? I though it was the id of the child device sending the alarm?

I though it was the id of the child device sending the alarm?

Yes it is. Not just for child devices, but even for the parent device.

I renamed extras as fragments

albinsuresh · 2023-01-31T06:20:35Z

crates/core/c8y_api/Cargo.toml

@@ -35,6 +35,7 @@ tracing = { version = "0.1", features = ["attributes", "log"] }
 anyhow = "1.0"
 assert-json-diff = "2.0"
 assert_matches = "1.5"
+maplit = "1.0.2"


This is fine as a dev dependency. But just FYI: you could statically create HashMap using the from method as follows:

let solar_distance = HashMap::from([ ("Mercury", 0.4), ("Venus", 0.7), ("Earth", 1.0), ("Mars", 1.5), ]);

crates/core/tedge_api/src/alarm.rs

albinsuresh · 2023-01-31T06:29:55Z

crates/core/tedge_api/src/alarm.rs

@@ -106,10 +126,18 @@ impl ThinEdgeAlarm {
                Some(serde_json::from_str(mqtt_payload)?)
            };

+            // The 4th part of the topic name is the event source - if any


Suggested change

// The 4th part of the topic name is the event source - if any

// The 4th part of the topic name is the alarm source - if any

albinsuresh · 2023-01-31T06:31:33Z

crates/core/tedge_mapper/src/c8y/alarm_converter.rs

+                    );
+                    output_messages.push(Message::new(&c8y_alarm_topic, smartrest_alarm));
+                } else {
+                    dbg!(&c8y_alarm);


Don't forget to remove this at the end.

didier-wenzek · 2023-02-01T08:56:11Z

crates/core/c8y_api/src/json_c8y.rs

+fn update_the_source(
+    source_name: &str,
+) -> Result<Option<HashMap<String, String>>, SMCumulocityMapperError> {


The name and the type signature of this function hardly convey its purpose.

I would simply return a hash.

Suggested change

fn update_the_source(

source_name: &str,

) -> Result<Option<HashMap<String, String>>, SMCumulocityMapperError> {

fn make_c8y_source_fragment(

source_name: &str,

) -> HashMap<String, String> {

didier-wenzek · 2023-02-01T08:58:14Z

crates/core/c8y_api/src/json_c8y.rs

+    #[serde(skip_serializing_if = "Option::is_none")]
+    #[serde(rename = "externalSource")]
+    pub source: Option<HashMap<String, String>>,


I would consider to remove the Option type, using an empty map instead of None.

I replaced the hash map with a SourceInfo struct for the source.

albinsuresh · 2023-02-01T09:13:35Z

crates/core/c8y_api/src/json_c8y.rs

 #[derive(Debug, Serialize, Deserialize, PartialEq, Eq)]
 #[serde(rename_all = "camelCase")]
 pub struct C8yCreateAlarm {
    #[serde(skip_serializing_if = "Option::is_none")]
-    pub source: Option<C8yManagedObject>,
+    #[serde(rename = "externalSource")]
+    pub source: Option<HashMap<String, String>>,


Instead of using a generic Hashmap here, I'd use a struct that replicates externalSource JSON structure:

"externalSource":{ "externalId":"<child-device-id>", "type":"c8y_Serial" }

reubenmiller · 2023-02-01T21:19:58Z

@PradeepKiruvale I found the reason why the alarm is not visible in the Cumulocity UI, the severity fields need to be in uppercase!

I just tested with your version here, and below is the json that I saw was being pushed to the c8y/alarm/alarms/create topic.

{
    "externalSource": {
        "externalId": "TST-extend_pure_loop_child1",
        "type": "c8y_Serial"
    },
    "severity": "critical",    // <<<<<<<<<<<<<<<<< needs to be UPPERCASE
    "type": "myCustomAlarmType",
    "time": "2023-02-01T21:08:14.800229538Z",
    "text": "Some test alarm",
    "someOtherCustomFragment": {
        "nested": {
            "value": "extra info"
        }
    }
}

PradeepKiruvale · 2023-02-02T03:33:10Z

@PradeepKiruvale I found the reason why the alarm is not visible in the Cumulocity UI, the severity fields need to be in uppercase!

I just tested with your version here, and below is the json that I saw was being pushed to the c8y/alarm/alarms/create topic.

{
    "externalSource": {
        "externalId": "TST-extend_pure_loop_child1",
        "type": "c8y_Serial"
    },
    "severity": "critical",    // <<<<<<<<<<<<<<<<< needs to be UPPERCASE
    "type": "myCustomAlarmType",
    "time": "2023-02-01T21:08:14.800229538Z",
    "text": "Some test alarm",
    "someOtherCustomFragment": {
        "nested": {
            "value": "extra info"
        }
    }
}

Good catch, but I remember testing with all CAPS. But still, it was not working then, Now it works, strange. Anyways I have updated the code to address this issue.

didier-wenzek

Approved

Signed-off-by: Pradeep Kumar K J <pradeepkumar.kj@softwareag.com>

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

Signed-off-by: Rina Fujino <18257209+rina23q@users.noreply.github.com>

chore: add maintainer scripts from existing debian packages for later comparison

…attempt Use JSON over MQTT to listen C8Y operations

PradeepKiruvale requested review from albinsuresh and reubenmiller January 30, 2023 07:02

didier-wenzek reviewed Jan 30, 2023

View reviewed changes

albinsuresh reviewed Jan 31, 2023

View reviewed changes

didier-wenzek reviewed Feb 1, 2023

View reviewed changes

albinsuresh reviewed Feb 1, 2023

View reviewed changes

reubenmiller added improvement User value theme:telemetry Theme: Telemetry data labels Feb 1, 2023

PradeepKiruvale force-pushed the feature/custom-frag-alarm branch from a6b539a to d40ed80 Compare February 2, 2023 03:37

didier-wenzek approved these changes Feb 2, 2023

View reviewed changes

albinsuresh approved these changes Feb 2, 2023

View reviewed changes

Alarms with custom fragments

d09703f

Signed-off-by: Pradeep Kumar K J <pradeepkumar.kj@softwareag.com>

PradeepKiruvale force-pushed the feature/custom-frag-alarm branch from e244f63 to d09703f Compare February 2, 2023 11:53

PradeepKiruvale merged commit 8d4ecbf into thin-edge:main Feb 2, 2023

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 2, 2023

Merge branch 'main' into add-robot-libraries

7617e38

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 2, 2023

add monitor step

f623796

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 2, 2023

remove message alarm field

49fac9a

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 2, 2023

remove todos

7a0b1c7

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

reubenmiller mentioned this pull request Feb 3, 2023

tedge alarm does not transform message fragment to text in the tedge-mapper-c8y #1710

Closed

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 3, 2023

support random device id

7fba498

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 3, 2023

split build and test jobs

673cf0d

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 3, 2023

change job names

01c52a1

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 3, 2023

improve job names

129bdcc

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 3, 2023

add cargo cache

d9ba943

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 4, 2023

change test package

0eacf05

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 4, 2023

update test deps

462c2ec

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request in reubenmiller/thin-edge.io Feb 5, 2023

remove docker network creation in workflow

e24078a

Signed-off-by: Reuben Miller <reuben.d.miller@gmail.com>

github-actions bot referenced this pull request Aug 6, 2023

tedge-configuration-plugin specification (#2097)

ab4e078

Signed-off-by: Rina Fujino <18257209+rina23q@users.noreply.github.com>

This was referenced Aug 7, 2023

chore: add maintainer scripts from existing debian packages for later comparison #2133

Merged

test(ci): reduce PR test report output by hiding passed tests #2134

Merged

github-actions bot referenced this pull request Aug 8, 2023

Merge pull request #2133 from reubenmiller/chore-add-maintainer-scripts

9518ad2

chore: add maintainer scripts from existing debian packages for later comparison

This was referenced Aug 8, 2023

Implement a converter that bridges the old MEA topics to the new ones #2125

Merged

chore: add c8y-configuration-plugin maintainer scripts in preparation for new packaging #2135

Merged

github-actions bot mentioned this pull request Sep 25, 2023

C8y mapping of entity registration messages #2266

Merged

18 tasks

This was referenced Nov 2, 2023

Add certificate based authentication to c8y-auth-proxy #2397

Closed

Add HTTPS/TLS support to the auth proxy #2430

Merged

github-actions bot mentioned this pull request Dec 8, 2023

Ordered processing of entity registration messages before any other messages #2466

Merged

18 tasks

github-actions bot mentioned this pull request Dec 18, 2023

Make entity store persistent #2428 #2522

Merged

11 tasks

github-actions bot mentioned this pull request Jan 9, 2024

chore: update system test dependencies #2577

Merged

11 tasks

github-actions bot referenced this pull request Jan 22, 2024

Merge pull request #2596 from rina23q/feature/1718/custom-fragment-re…

02d20ea

…attempt Use JSON over MQTT to listen C8Y operations

github-actions bot mentioned this pull request Jan 23, 2024

Limit dynamic operation dir APIs to main device #2409 #2614

Merged

11 tasks

github-actions bot mentioned this pull request Feb 23, 2024

Add support for built-in bridge to tedge-mapper-c8y #2716

Merged

11 tasks

github-actions bot mentioned this pull request Mar 18, 2024

fix: Honor c8y.mqtt setting on tedge connect #2789

Merged

13 tasks

github-actions bot mentioned this pull request May 6, 2024

Support different clouds in built-in bridge #2821

Merged

11 tasks

github-actions bot mentioned this pull request May 15, 2024

test: Fix remaining test failures from built-in bridge #2879

Merged

11 tasks

github-actions bot mentioned this pull request Jun 19, 2024

feat: use loopback ip address instead of alias to improve compatibility #2952

Merged

11 tasks

This was referenced Jul 17, 2024

fix: Handle messages republished by MQTT bridge following a disconnection event #2982

Closed

feat: use direct c8y connection for c8y-remote-access-plugin #3006

Merged

github-actions bot mentioned this pull request Aug 19, 2024

Add AWS keywords and create test for connecting thinedge to AWS using keys for authentication #3066

Closed

11 tasks

This was referenced Dec 5, 2024

feat: Entity registration HTTP API #3230

Draft

feat: Support multiple cloud profiles #3278

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support custom fragments in alarm creation #1699

Support custom fragments in alarm creation #1699

PradeepKiruvale commented Jan 30, 2023

reubenmiller commented Jan 30, 2023

didier-wenzek Jan 30, 2023

PradeepKiruvale Jan 31, 2023

didier-wenzek Jan 31, 2023

albinsuresh Feb 1, 2023

PradeepKiruvale Feb 2, 2023

albinsuresh Jan 31, 2023

albinsuresh Jan 31, 2023

PradeepKiruvale Jan 31, 2023

albinsuresh Jan 31, 2023

PradeepKiruvale Jan 31, 2023

didier-wenzek Feb 1, 2023

PradeepKiruvale Feb 2, 2023

didier-wenzek Feb 1, 2023

PradeepKiruvale Feb 2, 2023

albinsuresh Feb 1, 2023

PradeepKiruvale Feb 2, 2023

reubenmiller commented Feb 1, 2023

PradeepKiruvale commented Feb 2, 2023

didier-wenzek left a comment

	// The 4th part of the topic name is the event source - if any
	// The 4th part of the topic name is the alarm source - if any

Support custom fragments in alarm creation #1699

Support custom fragments in alarm creation #1699

Conversation

PradeepKiruvale commented Jan 30, 2023

Proposed changes

Types of changes

Paste Link to the issue

Checklist

Further comments

reubenmiller commented Jan 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reubenmiller commented Feb 1, 2023

PradeepKiruvale commented Feb 2, 2023

didier-wenzek left a comment

Choose a reason for hiding this comment