OSCORE support in Leshan

[sbernard31]

This issue aims to centralize all about OSCORE integration in Leshan 2.0.0(LWM2M 1.1).
Currently work is in progress leading by @rikard-sics, he also works on OSCORE integration in Californium.

See specification for more details :

• RFC8613 : Object Security for Constrained RESTful Environments (OSCORE)
• OSCORE in LWM2M 1.1
• end-to-end-security-for-the-internet-of-things page

The code will be available in a oscore branch, waiting we have a minimal viable feature which could be integrated in a 2.0.0 branch.

A minimal viable feature could be :

• OSCORE over coap at client side (see OSCORE over coap at client side #726)
• OSCORE over coap at server side (see OSCORE over coap at server side #727)

(Demo are not mandatory for a minimal viable feature but integration test should be there)

[rikard-sics]

Tentative plan for next steps:

1. Update code to use Californium 2.0.0-M17
2. Implement an Identity and related things in SecurityChecker for OSCORE
3. Work on the server side issues according to discussion in OSCORE over coap at server side #727
4. Add possibility of entering OSCORE context information as command line parameters for client

[rikard-sics]

I have now created PR #749 to update the code to use Californium 2.0.0-M17 instead. I will create a new PR to change the OSCORE context database to not be a singleton in Leshan anymore.

[rikard-sics]

I have now made a PR that adds support for setting OSCORE related security parameters in command line arguments to the client here #755

[sbernard31]

I add this link : https://omaspecworks.org/end-to-end-security-for-the-internet-of-things/ about this topic.

[rikard-sics]

I add this link : https://omaspecworks.org/end-to-end-security-for-the-internet-of-things/ about this topic.

Thanks for providing a link to that document. Nice to see it also brings up ACE and EDHOC. I am following the work in the IETF LAKE working group (for creating a lightweight authenticated key exchange for OSCORE). ACE is also something we are using in various use cases.

[rikard-sics]

Continuing the discussion from the last PR.

On a very high level the initial goal for me would be support for communicating with OSCORE between Client & Device Manager, communicating with OSCORE between Client & Bootstrap Server, and a Client being able receive bootstrapping information with OSCORE security material from the Bootstrap Server. Currently the first two points functionally work (but needs adjustments).

As a bit more detailed plan these are points I can think of:

• Rebase the current code on master
• Remove the ID Context input on client, Device Manager and Bootstrap Server
• Design and implement an OSCORE Store
• Implement bootstrapping to receive OSCORE security material

In the end I think it's better to rebase the code now, otherwise it would be even more potential conflicts in the future. As you mentioned in the last PR we should also discuss considerations related to the OSCORE store. I could create a separate issue about that.

[sbernard31]

In the end I think it's better to rebase the code now, otherwise it would be even more potential conflicts in the future.

About rebasing, maybe in practice it would be easier to cherry pick only needed commits (probably avoiding all which is not about OSCORE like SenML and californium integration?).
Even with this, I bet this will be a complicated task because a lot of code changed and some classes was moved.
Finally this is maybe even easier to just "rewrite"(copy/paste) the code.

As this will be really painful and as I feel myself a bit responsible for that. I would try to work on this.
(unless you already find a magic way to do that ?)

As you mentioned in the last PR we should also discuss considerations related to the OSCORE store. I could create a separate issue about that.

👍

[sbernard31]

@rikard-sics I started to work on rebasing, I create a dedicated issue to discuss about that : #866

[rikard-sics]

I have now created a PR to remove the ID context inputs as we discussed. See #867

[rikard-sics]

Some further comments now after the rebase. One thought I had was that it may be good to add some JUnit tests for OSCORE functionality?

Then I also realized an inconsistency. The way that Sender ID and Recipient ID are entered on the web UI for the Bootstrap Server and Management Server are opposite. Probably since the UI is for adding and configuring a new client it is better if the Sender ID and Recipient ID are from the client's point of view. I can make a change like that in the future.

So some further steps can be:

• Fix TODOs in the code
• Add JUnit tests
• Have consistent way in web UIs for adding Sender/Recipient ID

Also I will be fairly inactive now during August since I will be going on vacation.

[sbernard31]

So some further steps can be:

• Fix TODOs in the code
• Add JUnit tests
• Have consistent way in web UIs for adding Sender/Recipient ID

It sounds good to me.

Also I will be fairly inactive now during August since I will be going on vacation.

Me too at least the first 2 weeks of August :)

[rikard-sics]

Just to summarize the lists of next steps above with the points taken care of removed:

• Rebase the current code on master
• Remove the ID Context input on client, Device Manager and Bootstrap Server
• Design and implement an OSCORE Store
• Implement bootstrapping to receive OSCORE security material (pending)
• Fix TODOs in the code
• Add JUnit tests
• Have consistent way in web UIs for adding Sender/Recipient ID

[rikard-sics]

As you mentioned in the last PR we should also discuss considerations related to the OSCORE store. I could create a separate issue about that.

I have now created an issue for further discussion about an OSCORE store at #920.

[kamigor]

Hello Leshan Community.

Could anyone kindly help me to understand why I have faced the following fails when using server-client connections in OSCORE mode after the client is registered on the server.

1. None of the Objects (/1,/3,/6,/21,/3301) can be read via web interface. The 'Internal Server Error (unknown server)'
   message occurs above the Read button. No error messages in the server/client terminal. CoAP messages log for /6 Read: CON-POST + ACK-2.04
   Detailed run and configuration steps/logs are attached -
   leshan_oscore_run_configuration_steps.txt

2. All OSCORE object resources can be executed only (/21/12345/0...5)
   leshan_object_21_resources

3. After the client registration, the client account on the server's 'Security' tab contains 'Client Endpoint' field only. 'Security Mode' and 'Security Information' fields are disappeared.
   leshan_security_tab

[rikard-sics]

Could anyone kindly help me to understand why I have faced the following fails when using server-client connections in OSCORE mode after the client is registered on the server.

Hello. I have been working on adding OSCORE support to Leshan. Thank you for the feedback regarding this functionality. Let me try to reply to the points you list.

1. Yes unfortunately I am experiencing this also. Thanks for bringing this to my attention. It could be a regression introduced by some of the latest changes in the code or the rebasing on the master branch. I will look into this and try to solve it ASAP.

2. My understanding is that the OSCORE object on a client should not be readable from the LWM2M server. In fact it should probably not be in that list at all, even for Execute.

3. That should be because functionality to persist and present the OSCORE configuration for a client in the server web UI is currently missing. That is a TODO I have noted locally and aim to fix in a future update.

[rikard-sics]

1. None of the Objects (/1,/3,/6,/21,/3301) can be read via web interface. The 'Internal Server Error (unknown server)'
   message occurs above the Read button. No error messages in the server/client terminal. CoAP messages log for /6 Read: CON-POST + ACK-2.04
   Detailed run and configuration steps/logs are attached -
   leshan_oscore_run_configuration_steps.txt

I have now submitted a pull request that should also solve this issue.

[kamigor]

Rikard,
thank you for the timely response and the fix! I checked it and confirm that Read/Write/Observe/Cancellation/Create/Delete operations work properly.

During the '*demo.jar' making process I faced the error on '[INFO] leshan - integration tests ......................... FAILURE [ 11.396 s]' step. But I succeed the making with '-fn' option. The log for debugging purpose with '-e -X' is here.
mvn_install.zip

You mentioned that the OSCORE mode was added for DM account on the Bootstrap server and actually I could add it but NO_SEC mode is shown (due to it is not implemented yet). Is there a way to perform the Bootstrapping on the current stage from command line?
bootstrap_web_interface_dm_oscore

[kamigor]

Please ignore my question concerning the ability of the Bootstrap procedure. The client was successfully bootstrapped and reachable on DM server. The only issue is this 'Failed to retrieve OSCORE object linked from BS security object' line. Is it acceptable?

BS: java -jar leshan-bsserver-demo/target/leshan-bsserver-demo--SNAPSHOT-jar-with-dependencies.jar -lp 5693 -slp 5694
DM: java -jar leshan-server-demo/target/leshan-server-demo--SNAPSHOT-jar-with-dependencies.jar -wp 8081
Client: java -jar leshan-client-demo/target/leshan-client-demo-*-SNAPSHOT-jar-with-dependencies.jar -n oscore_client -b -u localhost:5693

2020-12-22 15:52:03,896 INFO LeshanClient - Starting Leshan client ...
2020-12-22 15:52:03,899 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:03,900 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:03,901 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:03,901 INFO LeshanClient - Leshan client[endpoint:oscore_client] started.
2020-12-22 15:52:03,901 INFO DefaultRegistrationEngine - Trying to start bootstrap session to coap://localhost:5693 ...
2020-12-22 15:52:03,958 INFO CaliforniumEndpointsManager - New endpoint created for server coap://localhost:5693 at coap://0.0.0.0:39767
2020-12-22 15:52:03,990 INFO DefaultRegistrationEngine - Bootstrap started
2020-12-22 15:52:04,004 DEBUG Security - Write on Security resource /0/0/0
....
2020-12-22 15:52:04,006 DEBUG Security - Write on Security resource /0/0/12
2020-12-22 15:52:04,017 DEBUG Security - Write on Security resource /0/1/0
....
2020-12-22 15:52:04,019 DEBUG Security - Write on Security resource /0/1/12
2020-12-22 15:52:04,019 DEBUG Security - Write on Security resource /0/1/17
2020-12-22 15:52:04,024 DEBUG Server - Write on Server resource /1/0/0
2020-12-22 15:52:04,025 DEBUG Server - Write on Server resource /1/0/1
2020-12-22 15:52:04,025 DEBUG Server - Write on Server resource /1/0/2
2020-12-22 15:52:04,025 DEBUG Server - Write on Server resource /1/0/6
2020-12-22 15:52:04,026 DEBUG Server - Write on Server resource /1/0/7
2020-12-22 15:52:04,029 DEBUG Security - Write on resource 0: LwM2mSingleResource [id=0, value=0102030405060708090a0b0c0d0e0f10, type=STRING]
2020-12-22 15:52:04,030 DEBUG Security - Write on resource 1: LwM2mSingleResource [id=1, value=01, type=STRING]
2020-12-22 15:52:04,030 DEBUG Security - Write on resource 2: LwM2mSingleResource [id=2, value=02, type=STRING]
2020-12-22 15:52:04,030 DEBUG Security - Write on resource 3: LwM2mSingleResource [id=3, value=10, type=INTEGER]
2020-12-22 15:52:04,030 DEBUG Security - Write on resource 4: LwM2mSingleResource [id=4, value=-10, type=INTEGER]
2020-12-22 15:52:04,031 DEBUG Security - Write on resource 5: LwM2mSingleResource [id=5, value=9e7ca92223786340, type=STRING]
2020-12-22 15:52:04,042 INFO DefaultRegistrationEngine - Bootstrap finished coap://localhost:5693.
2020-12-22 15:52:04,043 DEBUG Security - Read on resource 0
...
2020-12-22 15:52:04,043 DEBUG Security - Read on resource 5
2020-12-22 15:52:04,044 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:04,045 INFO CaliforniumEndpointsManager - Adding OSCORE context for coap://localhost:5683
2020-12-22 15:52:04,119 INFO CaliforniumEndpointsManager - New endpoint created for server coap://localhost:5683 at coap://0.0.0.0:54988
2020-12-22 15:52:04,119 DEBUG Security - Read on resource 0
...
2020-12-22 15:52:04,120 DEBUG Security - Read on resource 5
2020-12-22 15:52:04,120 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:04,120 INFO DefaultRegistrationEngine - Trying to register to coap://localhost:5683 ...
2020-12-22 15:52:04,147 INFO DefaultRegistrationEngine - Registered with location '/rd/h8JjIHYw3m'.
2020-12-22 15:52:04,148 INFO DefaultRegistrationEngine - Next registration update to coap://localhost:5683 in 53s...
2020-12-22 15:52:23,215 DEBUG Server - Read on Server resource /1/0/0
....
2020-12-22 15:52:23,217 DEBUG Server - Read on Server resource /1/0/23

[sbernard31]

2020-12-22 15:52:03,899 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:03,900 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object
2020-12-22 15:52:03,901 ERROR ServersInfoExtractor - Failed to retrieve OSCORE object linked from BS security object

I guess it could be an issue relative to #950 (comment).

'[INFO] leshan - integration tests ......................... FAILURE [ 11.396 s]' step.

I also face the tests failure I this this is because of : #950 (comment)

[rikard-sics]

The client was successfully bootstrapped and reachable on DM server. The only issue is this 'Failed to retrieve OSCORE object linked from BS security object' line. Is it acceptable?

Thanks for the feedback. I have now updated the PR and code in my branch to fix this.

[sbernard31]

@rikard-sics We didn't get news from you since a long time.
Hoping all is doing fine for you 🙂
Are you still working on OSCORE ?

[rikard-sics]

@rikard-sics We didn't get news from you since a long time.
Hoping all is doing fine for you
Are you still working on OSCORE ?

Hello. Yes I am indeed working on OSCORE. Currently I am working on implementing usage of the OSCORE Appendix B.2 procedure when OSCORE is used in Leshan. It is specified in the LWM2M 1.1 Transport Bindings document section 5.5.3 that Appendix B.2 of OSCORE should be used.

Basically Appendix B.2 derives a new OSCORE Security Context (with new Sender and Recipient keys). The benefit this has is that if a LWM2M client reboots and starts using the same Security Context that it was originally configured with, it will not be using the same Sender Key while starting over from sequence number 0 (thus having nonce and key reuse). But rather it will first run Appendix B.2 to generate a new Context (Sender and Recipient keys) with the LWM2M Server or LWM2M Bootstrap server. So essentially every time the client connects the first time using OSCORE to a LWM2M Server or LWM2M Bootstrap server, Appendix B.2 will be run. See https://tools.ietf.org/html/rfc8613#appendix-B.2

However, the core functionality for Appendix B.2 is implemented in Californium. While trying to make use of this in Leshan I realized there was an issue in the Californium code, in the specific case the client takes initiative to run Appendix B.2 but the server is then the first to actually send a request afterwards (as will happen when bootstrapping or registering). So basically I am now working on the Californium code to fix this issue (and some other things about Appendix B.2). My aim is to have a PR created for Californium in the coming week. Then I will move over to implement this in Leshan (perhaps I can have an intermediate solution until Californium releases a new version).

One nice benefit of having this Appendix B.2 functionality-wise is also that currently if the client is restarted but the server is not, the server will complain about replayed messages. But since Appendix B.2 refreshes the security contexts this problem will no longer exist.

[sbernard31]

But unfortunately I have been very occupied with other tasks and have not had time to put on this. Let me reflect a bit on your last post and come back with a reply.

@rikard-sics, I get you are very busy and if you succeed to find some time please to not hesitate to give us some feedback.
If this is possible, could you also give us some visibility about when you will be able to work on this again ?

I list here some recent comments which probably need a feedback from you :

• OSCORE support in Leshan #725 (comment)
• OSCORE store for holding security material #920 (comment)

@Michal-Wadowski is already very involved on this and I will try to imply more too, hoping we will be able to get a Minimal viable OSCORE feature integrated in master.

I think a plan could be :

1. adding some minimal integration Tests OSCORE integration & unit tests #1180
2. then rebase on master (dropping all demos features)
3. Define more clearly "how to use OSCORE" in Leshan. (this should help for OSCORE store task)
4. Find a way about OSCORE store OSCORE store for holding security material #920 (comment) (this one will probably imply some modification in Californium but maybe we can first try to implement this in Leshan with a custom implementation of OSCoreCtxDB.
5. Clean the code / add more tests if needed.
6. then add support to demo again.
7. add redis support.

[Michal-Wadowski]

I just rebased oscore branch on current master in PR #1201.
Demo support is dropped.
So 1. and 2. are already done.

[rikard-sics]

@rikard-sics, I get you are very busy and if you succeed to find some time please to not hesitate to give us some feedback.
If this is possible, could you also give us some visibility about when you will be able to work on this again ?

Yes I apologize for the absence. I should now be coming into a period where I can put more time into this work. I think one of the key points is considering how the OSCORE store is to be implemented.

I also want to thank @Michal-Wadowski for his interest and contributions to this work, it is very appreciated.

[sbernard31]

I just rebased oscore branch on current master in PR #1201.

I integrated this (#1201 (comment))

Yes I apologize for the absence.

No problem 😉

I think one of the key points is considering how the OSCORE store is to be implemented.

I agree.
Do you have a clear idea about this ? or should we need more discussion ? (if needed let's discuss about it at #920)

[sbernard31]

I will start to work on re-implementing the demo part of OSCORE feature (#1203).
(I will not polish this but this should help to make it available quickly)

[rikard-sics]

I will start to work on re-implementing the demo part of OSCORE feature (#1203).
(I will not polish this but this should help to make it available quickly)

Thanks, sounds great!

[adamsero]

Hi, sorry for asking again, but do you have any OSCORE status update? I noticed the commit from a week ago, but do you know approximately when OSCORE will be available on Client/Server/Bootstrap Server?

[sbernard31]

I noticed the commit from a week ago, but do you know approximately when OSCORE will be available on Client/Server/Bootstrap Server?

I guess it depends what you mean by "OSCORE will be available on Client/Server/Bootstrap Server"
I hope we will get an experimental minimal viable feature soon (maybe in few week/month), at least this is the plan (#1222)
I try to help as mush as I can but I think it depends a lot of @rikard-sics schedule, so he should maybe have better approximation than me ?

But anyway, keep in mind that this first version will probably not be production ready.
In my experience this kind of feature rarely perfectly work at first try.
We will probably find unworking use case, bug, security issue and/or interoperability issue.

[adamsero]

Ok, thanks for the information 👍

[sbernard31]

@adamsero, eventually if you want to help on this topic, you can play with oscore branch (eventually oscore_bsserver_demo which contains #1254 which should be integrated in oscore branch soon)

The idea would be to search for issues in main use cases.

[sbernard31]

OSCORE experimental support in now integrated in master #1277.
This experimental support will be shipped in 2.0.0-M8.

I also updated supported features : https://github.com/eclipse/leshan/wiki/LWM2M-1.1-supported-features

[sbernard31]

I list some missing point about a minimal viable feature :

• OSCORE fallback use cases #1257
• OSCORE Californium API/behavior changes. #1231
• OSCORE Context lifetime. #1230

[sbernard31]

@rikard-sics, we don't get so much news from you since a long time. I really hope you're doing well. 🙂

Please do not hesitate to give us some news from you.
E.g. if you work on Leshan/Californium let us know on which part.
If you are busy on something else let us know too (with maybe some inputs when you think you will work again on it)
If you are dealing with some OSCORE issue (maybe relative to interoperability or specification itself), let us know.
And If one day, you step out of this contribution let us know too. :)

[rikard-sics]

@rikard-sics, we don't get so much news from you since a long time. I really hope you're doing well. slightly_smiling_face

Yes, sorry for the period of silence. I have been busy mostly with other unrelated things. As for Californium I have prepared code for updating the Appendix B.2 functionality, after doing interop testing with another implementer. I intend to create a PR for that to Californium in the near future.

Currently I am working on wrapping up a paper, when that is done (a matter of weeks) I hope to have more time to get back to the work related to Leshan.

[sbernard31]

@rikard-sics, just to let you know that we are experimenting java-coap (fork) as californium alternative. (#1373)
Just in case, you will be interested on looking at this library and see if its design could make easier to integrate OSCORE feature.

By the way, @szysas does OSCORE (RFC8613) could be part of java-coap scope ? (this is a more a long term question)

[szysas]

Yes, it could be, but so far no work has been done nor started.

[rikard-sics]

@rikard-sics, just to let you know that we are experimenting java-coap (fork) as californium alternative. (#1373)
Just in case, you will be interested on looking at this library and see if its design could make easier to integrate OSCORE feature.

I see, thanks for the update. I will have a look at that library.