Support Cumulocity IoT basic device registration to allow SmartREST 1.0 usage for older clients

[reubenmiller]

Is your feature request related to a problem? Please describe.

Support Cumulocity IoT basic auth registration process to provide an upgrade path for existing customers whom use (legacy) SmartREST 1.0 (which is not supported when using cert based authentication)

Background

Existing customers with existing agents would like to take advantage of thin-edge.io and slowing migrate custom components. However these agents sometimes use SmartREST 1.0 which is incompatible with the cert-based device credentials (the default authentication used by thin-edge.io).

Describe the solution you'd like

Add support for basic auth device credentials for the Cumulocity IoT mapper so that devices can be registered following the Device Registration mechanism.

Implementation details

When using basic auth device credentials

• Do not store Cumulocity device credentials in the tedge.toml as these should be protected like a certificate (and not mixed with other non-sensitive configuration)
• c8y proxy must use a Basic Auth ,e.g. Authorization: Basic <base64(username:password)>
• Don't subscribe to s/uat and s/dat topics as they are only supported when using cert-based device credentials and errors will be posted to s/e if they are subscribed to

Open questions

• How to test an MQTT connection as the JWT topics are not available - Should a very basic SmartREST 1.0 template be used instead?

Implementation plan

The feature implementation can be broken into the following parts:

1. Support new configuration sets the registration mode (e.g. use basic auth, or device credentials/jwt)
   • Activate the basic mode via tedge config set c8y.registration_mode "basic"
   • Allow a username and password to be set via the command line and store in a new file (owned by tedge and with 600 permissions)
2. Update components to support using either Bearer or Basic auth for communication with Cumulocity IoT REST API (see below for a list of components)
3. Support registration of SmartREST 1.0 templates via a new tedge config tedge config set c8y.smartrest1.templates "template1,template2"
4. Support the Cumulocity IoT device registration process using the Cumulocity IoT bootstrap credentials when calling tedge connect and the "basic" registration mode is active (there are some UX aspects here to consider).

Describe alternatives you've considered

Additional context

Check list for components that require an update of the Authorization header mechanisms:

• Cumulocity IoT proxy (e.g. the 127.0.0.1:8001/c8y service)
• c8y-remote-access-plugin supports basic auth
• tedge-mapper
• bridges support both bearer and basic auth
  • mosquitto bridge
  • built-in bridge

[reubenmiller]

A proof of concept (PoC) has been created to check the general compatibility of thin-edge.io and SmartREST 1.0 when using a basic auth device user, and first signs seems that they are compatible.

The PoC used hacked together Rust code where it reads the device username/password (mostly) from environment variables in order to make work, it is by no means intended to be a good reference guide, however it still might be hopeful to see which parts of the code need to be adjusted. PR is provided here: #3039

• PoC PR: POC: check compatibility of thin-edge.io with Cumulocity IoT basic auth device users and SmartREST 1.0 #3039
• SmartREST 1.0 system tests: https://github.com/thin-edge/thin-edge.io/pull/3039/files#diff-f3db80154f22df259153ca552c61d1356f404a35f0c2220abedd240dba142a9d

[rina23q]

How to test an MQTT connection as the JWT topics are not available - Should a very basic SmartREST 1.0 template be used instead?

In SmartREST 1.0, we can use 61 as request and check 70 as response. About predefined message IDs, refer to here.

[rina23q]

Gaps between the requirements and the PoC:

1. Credentials (username and password) are stored in tedge.toml, whilst the requirements is in a different location.
2. Need to clean up the hacks around JWT token topics and retrievals.
3. Need to clean up the hacks in mosquitto bridge file for SmartREST1.0 custom templates.
4. Missing tedge connect c8y connection check with Basic Auth.

I think 1) will the most time consumption as we don't have such mechanism now. It requires a small design level decision, which means it contains uncertainty in estimate.
2) and 3) should be straight-forward.
4) can be implemented in 1-2 days by using the build-in templates.

Excluding the device bootstrapping, I would say 1.5 weeks (including review process).

[didier-wenzek]

1. Credentials (username and password) are stored in tedge.toml, whilst the requirements is in a different location.

I think 1) will the most time consumption as we don't have such mechanism now. It requires a small design level decision, which means it contains uncertainty in estimate.

The password must not be stored in tedge.toml but in well-protected file, which path is given by the config. Exactly as for the private key associated to device certificate.

There is small issue for the device id which is currently stored in the device certificate. We will have to handle a specific case here (storing/reading the device from an external file or simply tedge.toml).

[rina23q]

The Rust code and Robot test are merged by #3196. However, the user guide is not added yet.

[rina23q]

How to connect to c8y with Basic Auth

Given that thin-edge is already installed in a target device.

1. Create a self-signed certificate (the certificate will be used only to read its common name as device.id)

2. Set the c8y.auth_mode to basic

tedge config set c8y.auth_mode "basic"

3. Give username and password to the credentials file. The location is /etc/tedge/credentials.toml by default. The path is configurable by c8y.credentials_path.

[c8y]
username = "t5678/octocat"
password = "abcd1234"

4. Run sudo tedge connect c8y

How to apply SmartREST1.0 template

1. Upload the SmartREST1.0 template to c8y.
   Check the robot test for the example template and how to upload to c8y.
   

   thin-edge.io/tests/RobotFramework/tests/cumulocity/smartrest_one/smartrest_one.robot

   Line 42 in c078d4b

   ... curl --max-time 15 -sf -XPOST http://127.0.0.1:8001/c8y/s -H "Content-Type: plain/text" -H "X-Id: ${TEMPLATE_XID}" --data "${SMART_REST_ONE_TEMPLATES}"

2. Add the template external ID to config.

tedge config set c8y.smartrest1.templates "{template_xid}"

3. Update bridge config and restart all services by sudo tedge reconnect c8y.

[reubenmiller]

System tests have been added to cover the functionality