Added OPAQUE text encoding, using base64

[davideicardi]

Signed-off-by: Davide Icardi davide.icardi@gmail.com

See #675

[davideicardi]

Some doubts:

• Do we need to handle null values?
• It is correct to assume that the value is a byte[]? Casting ((byte[])val) is safe?

[sbernard31]

Do we need to handle null values?

This is not possible to create a LwM2mSingleResource with a null value : see code

It is correct to assume that the value is a byte[]? Casting ((byte[])val) is safe?

The LwM2mResource javadoc says : "If getType() returns Type.OPAQUE, the value is a byte array."

So I think we're good :)

Thx for this, did you plan to push the decoder in this PR or in another one ?

[davideicardi]

I will now have a look at the decoder. I think to add it here if for you is ok.

[sbernard31]

I think to add it here if for you is ok.

👍

[davideicardi]

Added decoder and test. I have squashed all commits ...

Do you think that we need a try/catch inside the decoder?

[sbernard31]

Do you think that we need a try/catch inside the decoder?

I would say we should handle the case where the string is not a valid base64 payload and so raise a CodecException !

But the Base64 javadoc is not so verbose about that ...
(this is a copy paste from apache commons codec, we didn't want to depends to this jar only for 2 methods)

So reading the code, it seems that IllegalStateException could be raised ?
There is also a isBase64 method ...
So several solutions :

• if isBase64() ?
• try catch IllegalStateException ?
• try catch Exception ? (probably not a so good idea)

We should add some test with bad payload, we will see how base64 behave and you could try several solutions and choose the best one ?

[davideicardi]

I have tried to write a test with an input that cause the decode to throw but without success. It seems that when an invalid characters is found it simply ignore it or trasform it to a +.

I have added a test to ensure this behaviour, see text_decode_opaque_should_ignore_not_base64_chars.

We can probably use isBase64 and throw but not sure if it is a good idea, maybe the validation should be performed at an higher level...

In any case I'm open to suggestions, I will follow you ;-)

[sbernard31]

It seems that when an invalid characters is found it simply ignore it or transform it to a +

With this implementation, Invalid characters is ignored.
The "-" transform in "+" is about url safe variant. see rfc4648#section-5, by default this implementation is able to decode both standard and url safe alphabet. But we encode with standard alphabet.

As I was a bit lost with all of this ... I read the RFC4648.
And my understanding is that rfc recommends to :

• NOT ignoring non alphabet characters
• Do not accept missing padding character
• be strict about canonical form meaning that padding bit must be 0 !

By the way the LWM2M specification does not talk about url safe alphabet.

So a perfect solution should be to have a default base64 decoder which :

1. rejects url safe alphabet : A-BB should be rejected (the good one is A+BB)
2. rejects invalid character : 'A+B;B! should be rejected
3. rejects 1 bit in padding : ABC= should be rejected (the canonical form is ABA=)
4. rejects missing padding character : ABA should be rejected (missing pad char ABA=)

by raising a CodecException.
And if user want a different behavior it could change it. (we will see later how to do that)

1. It seems we can not do that with this implementation ...
2. is doable with isBase64 check...
3. it seems we can not do that with this implementation ...
4. it seems we can not do that with this implementation ...

So, we can :

• go for a first shot implementation with just 2. and searching to improve this in a second time by searching a more appropriate implementation.
• or we search a good implementations for our needs now.

[davideicardi]

Thanks! Good investigation!

If for you is ok I will proceed with:

go for a first shot implementation with just 2. and searching to improve this in a second time by searching a more appropriate implementation.

I will first call isBase64 and in case of false I will throw a CodecException.

Ok?

[sbernard31]

I will first call isBase64 and in case of false I will throw a CodecException.
Ok?

👍

[davideicardi]

Ok done! let met know if you see problems ...

[sbernard31]

Sounds good, I will integrate this in master. (I create a new issue about issue we discover here, see #678 )

[sbernard31]

Thx @davideicardi ! 🙏

[davideicardi]

Great! Thank you @sbernard31 !
There is a nightly build or something similar available? In other words there is an easy way to get the master/unstable version from a maven repo?

[sbernard31]

We have a jenkins build : https://hudson.eclipse.org/leshan/job/leshan/, but jars are not deployed ... 😞

But building jars locally is pretty easy, doing this you will have a leshan-*-SNAPSHOT available in your local maven repository and so you could test it easily just adding SNAPSHOT version to you dependencies.

Last point, if you need a release just let me know :)

[davideicardi]

The problem is that we have some build pipelines running with the Leshan dependency ... so building it locally can be a little difficult ... Taking it from a maven repo is easier...
If you are able to release a new version in one week or two I will appreciate, otherwise no problem, I will pubish it in some temporary repo 😉

[sbernard31]

If you are able to release a new version in one week or two I will appreciate

I will try, but maybe doing some validation before could be a good idea 🤔
(I would like to integrate this PR #657 too but I wait for a teammate review :) )

[sbernard31]

@davideicardi did you made some tests ?
I plan to release a 1.0.0-M11 today ?

[davideicardi]

Good news! I have used a local compiled version during these days and for now it is working fine. Green light for me!

[sbernard31]

You probably don't care but finally #657 will not be integrated in this release. (next one)

[davideicardi]

Yes, no problem for me.

I think that now we can close #675 issue right? we can leave #678 open for the enhancements ..