Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2 #1306

avtolstoy · 2017-04-23T18:35:05Z

Problem

Bootloader needs to be a dependency of some system-module.

Solution

This PR is based on feature/bootloader_dct branch and #1289.

Adds HAL_Core_Validate_Modules function
When initializing system-part2, apart from verifying user module, this PR additionally checks bootloader and if dependency or integrity check fails, the device is forced into safe mode.

Steps to Test

N/A

Example App

N/A

References

https://github.com/spark/firmware/tree/feature/bootloader_dct

Completeness

User is totes amazing for contributing!
Contributor has signed CLA (Info here)
Problem and Solution clearly stated
Run unit/integration/application tests on device
Added documentation
Added to CHANGELOG.md after merging (add links to docs and issues)

Enhancement

[PR #1306] Bootloader module dependency and integrity checks have been added to system-part2. If they fail, the device is forced into safe mode and a new bootloader will be OTA transferred to the device.

…ded dynamically

… down the boot process

…otloader

…loader

technobly · 2017-04-24T16:57:19Z

@avtolstoy to help the Cloud team develop the support for this feature in the Safe Mode Healer, would you please post an example of what the describe message would look like with the new bootloader dependency, and also provide binaries of the bootloader, and system modules? That would be super helpful for testing.

system-part2 now depends both on system-part1 and bootloader

avtolstoy · 2017-04-26T08:14:05Z

Binaries: https://drive.google.com/drive/folders/0BySBnweX26fwRldJVGFuU0RJRU0?usp=sharing

Note: system-module binaries provided here are version 107, bootloader is 13

Bootloader is now a second dependency of system-part2 both on Electron and Photon/P1 (binary inspector probably needs support for this).

Electron

On develop (or 0.6.2-rc.1):

{
  "p": 10,
  "imei": "XXXXXXXXXXXXX",
  "iccid": "YYYYYYYYYYYYYYY",
  "m": [
    {
      "s": 16384,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "b",
      "n": "0",
      "v": 12,
      "d": []
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "1",
      "v": 106,
      "d": [
        {
          "f": "s",
          "n": "3",
          "v": 106,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "2",
      "v": 106,
      "d": [
        {
          "f": "s",
          "n": "1",
          "v": 106,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "3",
      "v": 106,
      "d": []
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "u": "25DC0C3F1769382A0494BE93AC6491F5918DB252912CC0E91B29BD28D27C44CE",
      "f": "u",
      "n": "1",
      "v": 4,
      "d": [
        {
          "f": "s",
          "n": "2",
          "v": 106,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "f",
      "vc": 30,
      "vv": 0,
      "d": []
    }
  ]
}

After updating system-part1, system-part2, system-part3, but not bootloader (device is in safe mode):

{
  "p": 10,
  "imei": "XXXXXXXXXXX",
  "iccid": "YYYYYYYYYYYYYYY",
  "m": [
    {
      "s": 16384,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "b",
      "n": "0",
      "v": 12,
      "d": []
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "1",
      "v": 107,
      "d": [
        {
          "f": "s",
          "n": "3",
          "v": 107,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 26,
      "f": "s",
      "n": "2",
      "v": 107,
      "d": [
        {
          "f": "s",
          "n": "1",
          "v": 107,
          "_": ""
        },
        {
          "f": "b",
          "n": "0",
          "v": 13,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "3",
      "v": 107,
      "d": []
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "u": "3FE3D246AB659DC01C4CFE030B89EE23FFBF54EDBAA402957CBC47078E631341",
      "f": "u",
      "n": "1",
      "v": 4,
      "d": [
        {
          "f": "s",
          "n": "2",
          "v": 107,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "f",
      "vc": 30,
      "vv": 0,
      "d": []
    }
  ]
}

Platform: 10 - Electron
Modules
  Bootloader module #0 - version 12, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 107, main location, 131072 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #3 - version 107
  System module #2 - version 107, main location, 131072 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: FAIL
      System module #1 - version 107
      Bootloader module #0 - version 13
  System module #3 - version 107, main location, 131072 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  User module #1 - version 4, main location, 131072 bytes max size
    UUID: 3FE3D246AB659DC01C4CFE030B89EE23FFBF54EDBAA402957CBC47078E631341
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 107
  empty - factory location, 131072 bytes max size

Updated bootloader:

{
  "p": 10,
  "imei": "XXXXXXXXX",
  "iccid": "YYYYYYYYYYY",
  "m": [
    {
      "s": 16384,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "b",
      "n": "0",
      "v": 13,
      "d": []
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "1",
      "v": 107,
      "d": [
        {
          "f": "s",
          "n": "3",
          "v": 107,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "2",
      "v": 107,
      "d": [
        {
          "f": "s",
          "n": "1",
          "v": 107,
          "_": ""
        },
        {
          "f": "b",
          "n": "0",
          "v": 13,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "f": "s",
      "n": "3",
      "v": 107,
      "d": []
    },
    {
      "s": 131072,
      "l": "m",
      "vc": 30,
      "vv": 30,
      "u": "3FE3D246AB659DC01C4CFE030B89EE23FFBF54EDBAA402957CBC47078E631341",
      "f": "u",
      "n": "1",
      "v": 4,
      "d": [
        {
          "f": "s",
          "n": "2",
          "v": 107,
          "_": ""
        }
      ]
    },
    {
      "s": 131072,
      "l": "f",
      "vc": 30,
      "vv": 0,
      "d": []
    }
  ]
}

Platform: 10 - Electron
Modules
  Bootloader module #0 - version 13, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 107, main location, 131072 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #3 - version 107
  System module #2 - version 107, main location, 131072 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 107
      Bootloader module #0 - version 13
  System module #3 - version 107, main location, 131072 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  User module #1 - version 4, main location, 131072 bytes max size
    UUID: 3FE3D246AB659DC01C4CFE030B89EE23FFBF54EDBAA402957CBC47078E631341
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 107
  empty - factory location, 131072 bytes max size

Photon

On develop (or 0.6.2-rc.1):

{"p":6,"m":[{"s":16384,"l":"m","vc":30,"vv":30,"f":"b","n":"0","v":12,"d":[]},{"s":262144,"l":"m","vc":30,"vv":30,"f":"s","n":"1","v":106,"d":[]},{"s":262144,"l":"m","vc":30,"vv":30,"f":"s","n":"2","v":106,"d":[{"f":"s","n":"1","v":106,"_":""}]},{"s":131072,"l":"m","vc":30,"vv":30,"u":"26488F71FF958591C887EA4609F3F70A0B067544977EDFBBF4EAD2656070394E","f":"u","n":"1","v":4,"d":[{"f":"s","n":"2","v":106,"_":""}]},{"s":131072,"l":"f","vc":30,"vv":0,"d":[]}]}

After updating system-part1, system-part2, but not bootloader (device is in safe mode):

{"p":6,"m":[{"s":16384,"l":"m","vc":30,"vv":30,"f":"b","n":"0","v":12,"d":[]},{"s":262144,"l":"m","vc":30,"vv":30,"f":"s","n":"1","v":107,"d":[]},{"s":262144,"l":"m","vc":30,"vv":26,"f":"s","n":"2","v":107,"d":[{"f":"s","n":"1","v":107,"_":""},{"f":"b","n":"0","v":13,"_":""}]},{"s":131072,"l":"m","vc":30,"vv":30,"u":"93E33C9310485C2E5D2390FBA9B101B3A8168EABA77AFC99C890CC53E596E265","f":"u","n":"1","v":4,"d":[{"f":"s","n":"2","v":107,"_":""}]},{"s":131072,"l":"f","vc":30,"vv":0,"d":[]}]}

Platform: 6 - Photon
Modules
  Bootloader module #0 - version 12, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 107, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #2 - version 107, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: FAIL
      System module #1 - version 107
      Bootloader module #0 - version 13
  User module #1 - version 4, main location, 131072 bytes max size
    UUID: 93E33C9310485C2E5D2390FBA9B101B3A8168EABA77AFC99C890CC53E596E265
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 107
  empty - factory location, 131072 bytes max size

Updated bootloader:

{"p":6,"m":[{"s":16384,"l":"m","vc":30,"vv":30,"f":"b","n":"0","v":13,"d":[]},{"s":262144,"l":"m","vc":30,"vv":30,"f":"s","n":"1","v":107,"d":[]},{"s":262144,"l":"m","vc":30,"vv":30,"f":"s","n":"2","v":107,"d":[{"f":"s","n":"1","v":107,"_":""},{"f":"b","n":"0","v":13,"_":""}]},{"s":131072,"l":"m","vc":30,"vv":30,"u":"93E33C9310485C2E5D2390FBA9B101B3A8168EABA77AFC99C890CC53E596E265","f":"u","n":"1","v":4,"d":[{"f":"s","n":"2","v":107,"_":""}]},{"s":131072,"l":"f","vc":30,"vv":0,"d":[]}]}

Platform: 6 - Photon
Modules
  Bootloader module #0 - version 13, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 107, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #2 - version 107, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 107
      Bootloader module #0 - version 13
  User module #1 - version 4, main location, 131072 bytes max size
    UUID: 93E33C9310485C2E5D2390FBA9B101B3A8168EABA77AFC99C890CC53E596E265
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 107
  empty - factory location, 131072 bytes max size

suda · 2017-04-26T08:39:44Z

Thanks Andrey! I see that bootloader shows up in the dependencies when running inspector and I also created a test case for binary-version-reader: particle-iot/binary-version-reader#15 so that part should be sorted. I'll move to testing SMH itself

m-mcgowan · 2017-04-26T09:58:42Z

Rather than adding a 2nd dependency, why not make the bootloader the root dependency (a dependency of system-part1 on the photon.)

avtolstoy · 2017-04-26T13:23:05Z

Just to duplicate what we discussed earlier: it is not possible to have bootloader as a dependency of system-part1 as I initially intended, as it will fail the dependency check when upgrading OTA/Ymodem. Whereas if we add it as a secondary dependency of system-part2, which is flashed last, we can safely let SMH upgrade the bootloader for us.

suda · 2017-04-27T13:13:27Z

Second check in SMH itself also confirms this should work: https://github.com/spark/workerd/pull/70

Once we have a ready bootloader image (I assumed the one Andrey posted isn't final yet), we can add it to the database and test it live.

technobly · 2017-04-27T14:30:55Z

Just an FYI, 0.6.2-rc.2 will be v107, and v0.7.0-rc.1 will be v200.

technobly · 2017-05-09T17:34:14Z

bootloader/src/photon/bootloader_dct.c

+#define MODULAR_FIRMWARE 1
+#include "../../../hal/src/photon/ota_module_bounds.c"
+
+#define SYSTEM_PART2_MIN_MODULE_VERSION 107 // 0.7.0-rc.1


v0.7.0-rc.1 will start at module version 200

Fixed in ca75375

technobly · 2017-05-09T18:51:24Z

hal/src/stm32f2xx/core_hal_stm32f2xx.c

+    const module_bounds_t* bounds = NULL;
+    hal_module_t mod;
+    bool module_fetched = false;
+    bool valid = true;


Should we initialize valid = false in case of a rogue PC jump over the actual valid assignment below?

This exists in hal/src/stm32f2xx/ota_flash_hal_stm32f2xx.cpp:92 as well.

Fixed in eedde55, however the default in validate_module_dependencies_full has to be left true.

…pendency Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2

sergeuz and others added 18 commits April 14, 2017 02:19

Load DCT functions dynamically in bootloader (P1)

4c1c306

Compilation fixes, renamings

835bcdb

Panic messages are now logged only in debug builds

f7ed26b

Disable reading/writing of DCT via DFU if DCT functions cannot be loa…

09d0351

…ded dynamically

Clear module info in DCT before updating modules

70c9e2a

Load DCT functions on first call; error handling for DCT read operations

66f222e

Use backup registers to store system flags (stm32f2xx)

cf3ca06

Check module dependencies

ead398b

Enable dynamic loading of the DCT functions on Photon

56c6158

Check addresses of the dynamically loaded functions

c97d7e0

Check IWDG flag in DCT for compatibility with old bootloaders

c12d5a6

Don't clear module info slots in DCT unconditionally to avoid slowing…

3e949ab

… down the boot process

Fix buttons setup in a situation when DCT functions are missing in bo…

c0f1e62

…otloader

Fill DFU packet with zeros to make reading errors more apparent

57081e2

Minor fixes

5d91aaf

Moves BOOTLOADER_VERSION definition into system_module_version.mk

403c980

Makes photon/p1 system-part1 and electron system-part3 depend on boot…

a1d5e17

…loader

Adds bootloader module validation

d306841

Adds a secondary module dependency.

b2e7a32

system-part2 now depends both on system-part1 and bootloader

Fixes unit tests. sizeof(void*) is 8 on 64-bit systems

5a1d198

avtolstoy changed the title ~~Adds bootloader as a dependency to Photon/P1 system-part1 and Electron system-part3~~ Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2 Apr 26, 2017

Add MODULE_DEPENDENCY2 to main/build.mk

86921a6

technobly added the enhancement label May 9, 2017

technobly added this to the 0.7.0 milestone May 9, 2017

technobly requested changes May 9, 2017

View reviewed changes

Default to invalid in HAL_Core_Validate_Modules

eedde55

technobly approved these changes May 10, 2017

View reviewed changes

technobly merged commit 9a93f40 into feature/photon/wiced-3.7.0-7 May 10, 2017

technobly deleted the feature/bootloader-dependency branch May 10, 2017 18:14

avtolstoy pushed a commit to avtolstoy/firmware that referenced this pull request May 10, 2017

Merge pull request particle-iot#1306 from spark/feature/bootloader-de…

de94801

…pendency Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2 #1306

Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2 #1306

avtolstoy commented Apr 23, 2017 •

edited by technobly

Loading

technobly commented Apr 24, 2017

avtolstoy commented Apr 26, 2017 •

edited by suda

Loading

suda commented Apr 26, 2017 •

edited

Loading

m-mcgowan commented Apr 26, 2017

avtolstoy commented Apr 26, 2017 •

edited

Loading

suda commented Apr 27, 2017

technobly commented Apr 27, 2017

technobly May 9, 2017

avtolstoy May 10, 2017

technobly May 9, 2017

avtolstoy May 10, 2017

Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2 #1306

Adds bootloader as a secondary dependency to Photon/P1/Electron system-part2 #1306

Conversation

avtolstoy commented Apr 23, 2017 • edited by technobly Loading

Problem

Solution

Steps to Test

Example App

References

Completeness

Enhancement

technobly commented Apr 24, 2017

avtolstoy commented Apr 26, 2017 • edited by suda Loading

Electron

Photon

suda commented Apr 26, 2017 • edited Loading

m-mcgowan commented Apr 26, 2017

avtolstoy commented Apr 26, 2017 • edited Loading

suda commented Apr 27, 2017

technobly commented Apr 27, 2017

technobly May 9, 2017

Choose a reason for hiding this comment

avtolstoy May 10, 2017

Choose a reason for hiding this comment

technobly May 9, 2017

Choose a reason for hiding this comment

avtolstoy May 10, 2017

Choose a reason for hiding this comment

avtolstoy commented Apr 23, 2017 •

edited by technobly

Loading

avtolstoy commented Apr 26, 2017 •

edited by suda

Loading

suda commented Apr 26, 2017 •

edited

Loading

avtolstoy commented Apr 26, 2017 •

edited

Loading