[MATTER-135] TE8 fixes (#16721)

OTA
* add support for compiling without OTA support. If this happens, then the
application is written at address 0 in flash (instead at 0 + 16K - required
for SSBL). Also, more space is available for the app in this case;
* add detailed README instructions;
* increase idle task stack size - that's because writes to the eeprom are
done on the idle task;
* allow the application to specify the block size for each OTA fragment;
* add NXP SDK fixes for fixing eeprom race conditions;
* add NXP SDK fix for allowing the processing of OTA blocks of different
sizes;
* add support for parsing the OTA header;
* fix OTA logic such that the application can pace the rate at which new OTA
fragments are requested;

Tokenizer
* add detailed README instructions.

Light app/Lock app
* enable lock tracking;
* schedule OTA/Matter attribute change/Server operations on the Matter task;
* avoid checking if thread is provisioned in a busy loop on eeprom - use an
event instead;
* disable mbedtls error logging (~13K) and instead add error logs in MemConfig.cpp
in case mbedtls memory allocations fail;
* change task priorities such that the app task has a smaller priority than the
Matter one;
* enable CHIP_DEVICE_CONFIG_ENABLE_TEST_SETUP_PARAMS such that the PBKDF are
hard-coded on the device side;
* refactor the gn options such that K32W0 module can be easily configured to be
attached to a dk different than DK6;
* group initialization calls in a single function;
* consider the case when WriteConfigValueStr is not called with a NULL-terminated
string.
* refactor board initialization code;
* allow building k32w light application with ota support - disable ble
for the moment, until a size-optimized SDK is released;
* fix NFC compilation error is BLE is disabled.

Signed-off-by: Doru Gucea <doru-cristian.gucea@nxp.com>

.github/.wordlist.txt

@@ -12,6 +12,7 @@ aabbccddeeff
 aarch
 abcdef
 abfb
+appwrite
 ABI
 ABIs
 ables
@@ -175,6 +176,7 @@ CatalogVendorId
 CBB
 cbd
 ccf
+csg
 CCMP
 CCS
 CCSTUDIO
@@ -315,6 +317,7 @@ dataset
 datasets
 DataVersion
 dbf
+doru
 DBG
 dBm
 DBUILD
@@ -594,8 +597,10 @@ IDF
 IDL
 IDLs
 idx
+ifconfig
 ifdef
 ifdefs
+itsfoss
 IGMP
 ignoreApplyUpdate
 ignoreQueryImage
@@ -897,6 +902,8 @@ optionOverride
 optionsMask
 optionsOverride
 orgs
+otaissues
+otatesting
 OTA
 OTADownloader
 otaDownloadPath
@@ -940,7 +947,10 @@ pbxproj
 PCA
 pcaps
 PDFs
+PDM
 PDK
+PSECT
+pdm
 peerAddrStr
 peerNodeId
 peerSessionId
@@ -949,6 +959,7 @@ percentageLiftValue
 periodicQueryTimeout
 pexpect
 pickString
+psect
 PID
 Pigweed
 PinCode
@@ -1019,6 +1030,7 @@ QRCodeUrl
 QSPI
 QueryImage
 QueryImageResponse
+Quickstart
 queryImageStatus
 qvCHIP
 RADVD
@@ -1028,6 +1040,7 @@ rAv
 RCP
 ReadAttribute
 ReadConfigValue
+RPis
 readelf
 readfds
 README
@@ -1089,6 +1102,7 @@ scalable
 scm
 sco
 scp
+ssbl
 ScriptBinding
 SDC
 SDHC
@@ -1119,6 +1133,7 @@ SetupQRCode
 sexualized
 SHA
 ShadeConfiguration
+SSBL
 showDocumentation
 shubhamdp
 SIGINT
@@ -1345,6 +1360,7 @@ whde
 whitespace
 whitespaces
 whl
+wg
 wic
 WiFiNetworkDiagnostics
 WindowCovering

.github/workflows/examples-k32w.yaml

@@ -70,8 +70,8 @@ jobs:
  run: |
  scripts/run_in_build_env.sh "\
  ./scripts/build/build_examples.py \
- --target k32w-light-release \
- --target k32w-light-tokenizer-release \
+ --target k32w-light-ota-se \
+ --target k32w-light-release-no-ota \
  --target k32w-lock-low-power-release \
  --target k32w-shell-release \
  build \
@@ -82,7 +82,7 @@ jobs:
  run: |
  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
  k32w k32w061+release light \
- out/artifacts/k32w-light-release/chip-k32w061-light-example \
+ out/artifacts/k32w-light-release-no-ota/chip-k32w061-light-example \
  /tmp/bloat_reports/
  - name: Get lock size stats
  timeout-minutes: 5

examples/lighting-app/nxp/k32w/k32w0/BUILD.gn

@@ -21,6 +21,7 @@ import("${k32w0_sdk_build_root}/k32w0_sdk.gni")
 
 import("${chip_root}/src/crypto/crypto.gni")
 import("${chip_root}/src/lib/core/core.gni")
+import("${chip_root}/src/lib/core/core.gni")
 import("${chip_root}/src/platform/device.gni")
 
 if (chip_pw_tokenizer_logging) {
@@ -82,7 +83,7 @@ k32w0_executable("light_app") {
  deps = [
  ":sdk",
  "${chip_root}/examples/common/QRCode",
- "${chip_root}/examples/ota-requestor-app/ota-requestor-common",
+ "${chip_root}/examples/lighting-app/lighting-common/",
  "${chip_root}/src/lib",
  "${chip_root}/third_party/mbedtls:mbedtls",
  "${k32w0_platform_dir}/app/support:freertos_mbedtls_utils",
@@ -108,6 +109,15 @@ k32w0_executable("light_app") {
 
  ldflags = [ "-T" + rebase_path(ldscript, root_build_dir) ]
 
+ if (chip_enable_ota_requestor) {
+ ldflags += [
+ "-Wl,--defsym",
+ "-Wl,__app_load_address__=0x4000",
+ "-Wl,--defsym",
+ "-Wl,__app_stated_size__=0x99A00",
+ ]
+ }
+
  output_dir = root_out_dir
 }
 

examples/lighting-app/nxp/k32w/k32w0/README.md

@@ -23,13 +23,14 @@ network.
 - [Device UI](#device-ui)
 - [Building](#building)
 - [Flashing and debugging](#flashdebug)
-- [Testing the example](#testing-the-example)
 - [Pigweed Tokenizer](#tokenizer)
  - [Detokenizer script](#detokenizer)
  - [Notes](#detokenizer-notes)
  - [Known issues](#detokenizer-known-issues)
-
-<hr>
+- [OTA](#ota) - [Writing the SSBL](#ssbl) - [Writing the PSECT](#psect) -
+ [Writing the application](#appwrite) - [OTA Testing](#otatesting) -
+ [Known issues](#otaissues)
+ </hr>
 
 <a name="intro"></a>
 
@@ -226,13 +227,6 @@ All you have to do is to replace the Openthread binaries from the above
 documentation with _out/debug/chip-k32w061-light-example.bin_ if DK6Programmer
 is used or with _out/debug/chip-k32w061-light-example_ if MCUXpresso is used.
 
-## Testing the example
-
-The app can be deployed against any generic OpenThread Border Router. See the
-guide
-[Commissioning NXP K32W using Android CHIPTool](../../../docs/guides/nxp_k32w_android_commissioning.md)
-for step-by-step instructions.
-
 <a name="tokenizer"></a>
 
 ## Pigweed tokenizer
@@ -299,3 +293,207 @@ script won't find them in the special-created sections.
 If run, closed and rerun with the serial option on the same serial port, the
 detokenization script will get stuck and not show any logs. The solution is to
 unplug and plug the board and then rerun the script.
+
+<a name="ota"></a>
+
+## OTA
+
+The internal flash needs to be prepared for the OTA process. First 16K of the
+internal flash needs to be populated with a Secondary Stage Bootloader (SSBL)
+related data while the last 8.5K of flash space is holding image directory
+related data (PSECT). The space between these two zones will be filled by the
+application.
+
+<a name="ssbl"></a>
+
+### Writing the SSBL
+
+The SSBL can ge generated from one of the SDK demo examples. The SDK demo
+example needs to be compiled inside MCUXpresso with the define _PDM_EXT_FLASH_.
+The SSBL demo application can be imported from the _Quickstart panel_: _Import
+SDK example(s)_ -> select _wireless->framework->ssbl_ application.
+
+![SSBL Application Select](../../../../platform/nxp/k32w/k32w0/doc/images/ssbl_select.JPG)
+
+The SSBL project must be compiled using the PDM_EXT_FLASH define.
+
+![PDM_EXT_FLASH](../../../../platform/nxp/k32w/k32w0/doc/images/pdm_ext_flash.JPG)
+
+Once compiled, the required ssbl file is called k32w061dk6_ssbl.bin
+
+![SSBL_BIN](../../../../platform/nxp/k32w/k32w0/doc/images/ssbl_bin.JPG)
+
+Before writing the SSBL, it it recommanded to fully erase the internal flash:
+
+```
+DK6Programmer.exe -V 5 -P 1000000 -s <COM_PORT> -e Flash
+```
+
+k32w061dk6_ssbl.bin must be written at address 0 in the internal flash:
+
+```
+DK6Programmer.exe -V2 -s <COM_PORT> -P 1000000 -Y -p FLASH@0x00="k32w061dk6_ssbl.bin"
+```
+
+<a name="psect"></a>
+
+### Writing the PSECT
+
+First, image directory 0 must be written:
+
+```
+DK6Programmer.exe -V5 -s <COM port> -P 1000000 -w image_dir_0=0000000010000000
+```
+
+Here is the interpretation of the fields:
+
+```
+00000000 -> start address 0x00000000
+1000 -> size = 0x0010 pages of 512-bytes (= 8kB)
+00 -> not bootable (only used by the SSBL to support SSBL update)
+00 -> SSBL Image Type
+```
+
+Second, image directory 1 must be written:
+
+```
+DK6Programmer.exe -V5 -s <COM port> -P 1000000 -w image_dir_1=00400000CD040101
+```
+
+Here is the interpretation of the fields:
+
+```
+00400000 -> start address 0x00004000
+CD04 -> 0x4CD pages of 512-bytes (= 614,5kB)
+01 -> bootable flag
+01 -> image type for the application
+```
+
+<a name="appwrite"></a>
+
+### Writing the application
+
+DK6Programmer can be used for flashing the application:
+
+```
+DK6Programmer.exe -V2 -s <COM_PORT> -P 1000000 -Y -p FLASH@0x4000="chip-k32w061-light-example.bin"
+```
+
+If debugging is needed, MCUXpresso can be used then for flashing the
+application. Please make sure that the application is written at address 0x4000:
+
+![FLASH_LOCATION](../../../../platform/nxp/k32w/k32w0/doc/images/flash_location.JPG)
+
+<a name="otatesting"></a>
+
+### OTA Testing
+
+The OTA topology used for OTA testing is illustrated in the figure below.
+Topology is similar with the one used for Matter Test Events.
+
+![OTA_TOPOLOGY](../../../../platform/nxp/k32w/k32w0/doc/images/ota_topology.JPG)
+
+The concept for OTA is the next one:
+
+- there is an OTA Provider Application that holds the OTA image. In our case,
+ this is a Linux application running on an Ubuntu based-system;
+- the OTA Requestor functionality is embedded inside the Lighting Application.
+ It will be used for requesting OTA blocks from the OTA Provider;
+- the controller (a linux application called chip-tool) will be used for
+ commissioning both the device and the OTA Provider App. The device will be
+ commissioned using the standard Matter flow (BLE + IEEE 802.15.4) while the
+ OTA Provider Application will be commissioned using the _onnetwork_ option
+ of chip-tool;
+- during commissioning, each device is assigned a node id by the chip-tool
+ (can be specified manually by the user). Using the node id of the device and
+ of the lighting application, chip-tool triggers the OTA transfer by invoking
+ the _announce-ota-provider_ command - basically, the OTA Requestor is
+ informed of the node id of the OTA Provider Application.
+
+_Computer #1_ can be any system running an Ubuntu distribution. We recommand
+using TE 7.5 instructions from
+[here](https://groups.csa-iot.org/wg/matter-csg/document/24839), where RPi 4 are
+proposed. Also, TE 7.5 instructions document point to the OS/Docker images that
+should be used on the RPis. For compatibility reasons, we recommand compiling
+chip-tool and OTA Provider applications with the same commit id that was used
+for compiling the Lighting Application. Also, please note that there is a single
+controller (chip-tool) running on Computer #1 which is used for commissioning
+both the device and the OTA Provider Application. If needed,
+[these instructions](https://itsfoss.com/connect-wifi-terminal-ubuntu/) could be
+used for connecting the RPis to WiFi.
+
+Build the Linux OTA provider application:
+
+```
+doru@computer1:~/connectedhomeip$ : ./scripts/examples/gn_build_example.sh examples/ota-provider-app/linux out/ota-provider-app chip_config_network_layer_ble=false
+```
+
+Build OTA image and start the OTA Provider Application:
+
+```
+doru@computer1:~/connectedhomeip$ : ./src/app/ota_image_tool.py create -v 0xDEAD -p 0xBEEF -vn 1 -vs "1.0" -da sha256 chip-k32w061-light-example.bin chip-k32w061-light-example.ota
+doru@computer1:~/connectedhomeip$ : rm -rf /tmp/chip_*
+doru@computer1:~/connectedhomeip$ : ./out/ota-provider-app/chip-ota-provider-app -f chip-k32w061-light-example.ota
+```
+
+Build Linux chip-tool:
+
+```
+doru@computer1:~/connectedhomeip$ : ./scripts/examples/gn_build_example.sh examples/chip-tool out/chip-tool-app
+```
+
+Provision the OTA provider application and assign node id _1_. Also, grant ACL
+entries to allow OTA requestors:
+
+```
+doru@computer1:~/connectedhomeip$ : rm -rf /tmp/chip_*
+doru@computer1:~/connectedhomeip$ : ./out/chip-tool-app/chip-tool pairing onnetwork 1 20202021
+doru@computer1:~/connectedhomeip$ : ./out/chip-tool-app/chip-tool accesscontrol write acl '[{"fabricIndex": 1, "privilege": 5, "authMode": 2, "subjects": [112233], "targets": null}, {"fabricIndex": 1, "privilege": 3, "authMode": 2, "subjects": null, "targets": null}]' 1 0
+```
+
+Provision the device and assign node id _2_:
+
+```
+doru@computer1:~/connectedhomeip$ : ./out/chip-tool-app/chip-tool pairing ble-thread 2 hex:<operationalDataset> 20202021 3840
+```
+
+Start the OTA process:
+
+```
+doru@computer1:~/connectedhomeip$ : ./out/chip-tool-app/chip-tool otasoftwareupdaterequestor announce-ota-provider 1 0 0 0 2 0
+```
+
+<a name="otaissues"></a>
+
+## Known issues
+
+- SRP cache on the openthread border router needs to flushed each time a new
+ commissioning process is attempted. For this, factory reset the device, then
+ execute _ot-ctl server disable_ followed by _ot-ctl server enable_. After
+ this step, the commissioning process of the device can start;
+- Due to some MDNS issues, the commissioning of the OTA Provider Application
+ may fail. Please make sure that the SRP cache is disabled (_ot-ctl srp
+ server disable_) on the openthread border router while commissioning the OTA
+ Provider Application;
+- No other Docker image should be running (e.g.: Docker image needed by Test
+ Harness) except the OTBR one. A docker image can be killed using the
+ command:
+
+```
+doru@computer1:~/connectedhomeip$ : sudo docker kill $container_id
+```
+
+- In order to avoid MDNS issues, only one interface should be active at one
+ time. E.g.: if WiFi is used then disable the Ethernet interface and also
+ disable multicast on that interface:
+
+```
+doru@computer1:~/connectedhomeip$ sudo ip link set dev eth0 down
+doru@computer1:~/connectedhomeip$ sudo ifconfig eth0 -multicast
+```
+
+- If OTBR Docker image is used, then the "-B" parameter should point to the
+ interface used for the backbone.
+
+- If Wi-Fi is used on a RPI4, then a 5Ghz network should be selected.
+ Otherwise, issues related to BLE-WiFi combo may appear.