-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sys/senml: add SenML modules #16384
sys/senml: add SenML modules #16384
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general, this looks nice. I'm not sure about the senml_saul
module though. I'm not sure many users will need it since it reads all saul_reg_t
by default, it seems like something that would be better fitted to be in the examples.
I think what would make more sense is having a:
void senml_saulreg_encode(void *ctx, saul_reg_t *dev);
ssize_t senml_saulregs_encode(coid *ctx, saul_reg_t **dev, uint8_t num);
Where the second would be an array of saul_regs_t
. I think in most cases the user would know how many sensors
it will encode and therefore a definite array is a better solution, a convenience wrapper could be provided as well, so something like:
ssize_t senml_saulregs_encode_cbor(uint8_t *buf, size_t len, saul_reg_t **dev, uint8_t num)
{
nanocbor_encoder_t enc;
nanocbor_encoder_init(&enc, buf, len);
nanocbor_fmt_array(&enc, num);
for (uint8_t i = 0; i < num; i++) {
senml_saulreg_encode(&enc, *dev++);
}
return nanocbor_encoded_len(&enc);
}
makefiles/pseudomodules.inc.mk
Outdated
@@ -125,6 +125,7 @@ PSEUDOMODULES += saul_pwm | |||
PSEUDOMODULES += scanf_float | |||
PSEUDOMODULES += sched_cb | |||
PSEUDOMODULES += semtech_loramac_rx | |||
PSEUDOMODULES += senml_% |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its better to use explicit modules, it at least keeps a clear list of the modules that are expected or not.
sys/include/senml.h
Outdated
senml_unit_t base_unit; /**< Base Unit */ | ||
senml_numeric_t base_value; /**< Base Value */ | ||
senml_numeric_t base_sum; /**< Base Sum */ | ||
uint64_t base_version; /**< Base Version */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it make sense to have compile-time configuration remove some of the optional fields and save some RAM?, specifically base_version
, update_time
, sum
, which will be used more rarely.
sys/include/senml.h
Outdated
* @param v Value to encode. | ||
* @return Numeric value containing the given value. | ||
*/ | ||
inline senml_numeric_t senml_float(float v) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
inline senml_numeric_t senml_float(float v) | |
static inline senml_numeric_t senml_float(float v) |
And for other cases below.
sys/include/senml.h
Outdated
* @param v Value to encode. | ||
* @return Numeric value containing the given value. | ||
*/ | ||
inline senml_numeric_t senml_float(float v) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we add static initializers as well? Some of the data structures are quite heavy (150bytes) and could be statically allocated to save stack space in many cases.
This is no longer waiting. Can you rebase @silkeh |
I agree. I would also like to see an API for individual devices. This would be very helpful. |
SAUL is structured in such a way that the dimensions of sensor values are entirely unknowable until you actually read it. I think it can even change on every read (theoretically). This means that the number of sensors and the number of SenML values have a ratio of somewhere between As for passing a list with a number of sensors: is it common to have a list in such a way? Why not just pass a Taking the above into account, I have split the implementation in two functions: int senml_saul_reg_encode_cbor(nanocbor_encoder_t *enc, saul_reg_t *dev);
size_t senml_saul_encode_cbor(uint8_t *buf, size_t len, saul_reg_t *reg); The first function encodes all dimensions of a SAUL sensor (and returns the number of dimensions encoded), and the second one is encodes all functions in a registry. The second one is extremely simple now, and I think it would be fine to move that to the example if so desired. Encoding all the temperature values would then look something like this: nanocbor_encoder_t enc;
nanocbor_encoder_init(&enc, buf, len);
nanocbor_fmt_array_indefinite(&enc);
saul_reg_t *dev = saul_reg;
while (dev) {
if(dev->driver->type = SAUL_SENSE_TEMP) {
senml_saul_reg_encode_cbor(&enc, dev);
}
dev = dev->next;
}
nanocbor_fmt_end_indefinite(&enc); |
I think a point you forget is that a user can also create a custom linked list of
For the second function: Why not having |
So why not create a custom registry instead of a linked list? The current implementation allows you to pass any nanocbor_encoder_t enc;
nanocbor_encoder_init(&enc, buf, len);
nanocbor_fmt_array(&enc, sensor_count);
for (int i = 0; i < sensor_count; i++) {
senml_saul_reg_encode_cbor(&enc, sensors[i]);
}
return nanocbor_encoded_len(&enc)
Sure, but just use something like the example above. The code is sufficiently composable to do whatever you like. |
I'm testing |
Nice then its only a stack issue |
Did the same thing or
|
So this patch was enough to fix compilation issues with diff --git a/tests/senml_cbor/Makefile b/tests/senml_cbor/Makefile
index 971ce780ee..abac3c6b6a 100644
--- a/tests/senml_cbor/Makefile
+++ b/tests/senml_cbor/Makefile
@@ -8,4 +8,6 @@ CFLAGS += -DCONFIG_SENML_ATTR_SUM
CFLAGS += -DCONFIG_SENML_ATTR_VERSION
CFLAGS += -DCONFIG_SENML_ATTR_UPDATE_TIME
+CFLAGS += -DTHREAD_STACKSIZE_DEFAULT=1536
+
include $(RIOTBASE)/Makefile.include
diff --git a/tests/senml_phydat/Makefile b/tests/senml_phydat/Makefile
index 6c14df0faa..452d330fda 100644
--- a/tests/senml_phydat/Makefile
+++ b/tests/senml_phydat/Makefile
@@ -2,5 +2,6 @@ include ../Makefile.tests_common
USEMODULE += senml_phydat
USEMODULE += embunit
+USEMODULE += printf_float
include $(RIOTBASE)/Makefile.include
diff --git a/tests/senml_phydat/main.c b/tests/senml_phydat/main.c
index 6cc295d96f..76b38bad66 100644
--- a/tests/senml_phydat/main.c
+++ b/tests/senml_phydat/main.c
@@ -28,6 +28,10 @@
#define ENABLE_DEBUG (0)
#include "debug.h"
+#ifndef PRIi64
+#define PRIi64 "lli"
+#endif
+
typedef struct {
phydat_t phydat;
senml_value_t senml; But still both avr and msp430 are failing with rounding floats:
What test fails depends on what |
This could be a nanocbor issue with AVR since the |
Maybe we should only run the test on platforms with known predictable rounding, eg: diff --git a/tests/senml_phydat/main.c b/tests/senml_phydat/main.c
index 76b38bad66..02209fe596 100644
--- a/tests/senml_phydat/main.c
+++ b/tests/senml_phydat/main.c
@@ -278,7 +278,10 @@ Test *tests_senml(void)
EMB_UNIT_TESTFIXTURES(fixtures) {
new_TestFixture(test_phydat_date_to_senml),
new_TestFixture(test_phydat_time_to_senml),
+/* Only run this test on CPUs with predictable rounding */
+#if defined(CPU_NATIVE) || defined(CPU_STM32)
new_TestFixture(test_phydat_to_senml_float),
+# endif
new_TestFixture(test_phydat_to_senml_decimal),
};
EMB_UNIT_TESTCALLER(senml_tests, NULL, NULL, fixtures); If we really want to test if floats/rounding works predictably/reliably on all platforms there should probably be a different test for that. |
I would rather go the other way around and not enable them for |
That's fine as well, although I'm having a bit of trouble finding what I should put in the |
For avr there is |
I have pushed three fixup commits:
Please take a look and feel free to squash. |
So it seems there is indeed an issue with |
I pushed a couple of nitpicks |
3f2d6aa
to
35de03a
Compare
Rebased |
35de03a
to
512a1c5
Compare
Squashed |
@chrysn Are you ok with getting this one in and addressing SenML/PhyDat unit issues separately? |
Yes, either sequence is OK, and this PR appears to be quite ready. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK, there are still some pending issues with:
-
issues with handling of doubles on AVR, but this seems to come from either nanocbor or the CPU itself
-
to limit builds jobs I would move all tests to a single application, this can be done in a follow-up, I have a branch with those somewhere (or a patch in the PR)
Other than this LGTM, I've tested on different architectures and BOARDs and except for the above mentioned issues and rounding differences on floats between platforms it all looks OK.
512a1c5
to
7db5992
Compare
Force pushed missing Kconfig in test and some blacklisting. |
Add a basic SenML module and submodules with support for: - Encoding SenML values as CBOR using NanoCBOR. - Converting from Phydat to SenML. - Reading and encoding SAUL sensors.
7db5992
to
9d61bdb
Compare
Wuhuuu! Thanks for this one @silkeh and thanks for sticking through! |
Yay! Thanks for all the help everyone! |
Contribution description
Add a basic SenML module with support for:
Some effort is made to convert units from those in Phydat to those supported by SenML, which results in some code that should probably not be part of a SenML implementation (including calculation of Unix timestamps).
Testing procedure
Includes two test cases and an example. The
senml_saul
test isn't useful at this point, but I couldn't find if there are some mocked sensors already.Issues/PRs references
Depends on bergzand/NanoCBOR#50