[Gen 3] 256KB application binary support when flashing over DFU

[avtolstoy]

Description

In preparation for 3.1.0-rc.1 Device OS release with support for 256KB (vs 128KB prior) application binaries, CLI needs to be updated to take destination address when performing DFU flashing from the binary module prefix, instead of using a hardcoded location.

This PR also bumps binary-version-reader dependency to the version with particle-iot/binary-version-reader#41, which improves how module prefixes are parsed.

I have not updated src/lib/deviceSpecs/specifications.js with the new location of the user application binary for Gen 3 platforms. This new location is only true for devices running >= 3.1.0, but it should not matter much anyway given that we are taking the destination address from the actual binary, which should cover both older (128KB) and newer (256KB) applications.

@busticated Any tests that we can update to validate this change?

How to Test

1. Use a Tracker and update it to 3.0.0 first, then flash 3.1.0-alpha.1: tracker@3.1.0-alpha.1.zip

particle usb listen
particle flash --serial tracker-bootloader@3.1.0-alpha.1.bin
particle usb dfu
particle flash --usb tracker-system-part1@3.1.0-alpha.1.bin

2. Attempt to flash attached tracker-tinker-large@3.1.0-alpha.1.bin with current version of CLI over DFU. It should fail with

dfu-util: Last page at 0x000fb2fb is not writeable

3. CLI from this branch should correctly flash this binary over DFU without an error
4. Put the device into listening mode and do serial inspect, it should report 256KB application (note the '262144 bytes max size)

  User module #2 - version 6, main location, 262144 bytes max size
    UUID: 85839E2B7CD45A4FDF7F28F7CD782914533923B13D8F5F8AB7C7561B832131E1
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 3100

5. Put the device into DFU mode and flash old tinker, it should work again just fine and the device should report 128KB application

$ particle usb dfu
$ particle flash --usb tinker
$ particle usb listen
$ particle serial inspect
...
  User module #1 - version 6, main location, 131072 bytes max size
    UUID: E48F6E1301CC55562FED63FFDF7BB756B4D043F0B7A894028827D9205AFB60A5
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 1514

Related Issues / Discussions

• [CH60338]
• [Gen 3] 256KB application support device-os#2322
• Fix HalModuleParser.parsePrefix() when parsing valid compressed binaries binary-version-reader#41

Completeness

• User is totes amazing for contributing!
• Contributor has signed CLA
• Problem and solution clearly stated
• Tests have been provided
• Docs have been updated
• CI is passing

[busticated]

thanks @avtolstoy 🙏

i'll aim to work through your testing instructions over the long weekend here.

Any tests that we can update to validate this change?

i'd think we could extend our e2e tests over here:

https://github.com/particle-iot/particle-cli/blob/feature/ch60338/gen-3-increasing-application-binary-size/test/e2e/flash.e2e.js#L170-L186

(see here for local setup instructions)

beyond those, maybe these legacy integration tests?

https://github.com/particle-iot/particle-cli/blob/feature/ch60338/gen-3-increasing-application-binary-size/test/integration/update-binaries.integration.js

...though the e2e tests are more helpful imo.

[avtolstoy]

@busticated Actually looking at that test I think it should already cover this particular change. We don't have to specifically validate 256KB Gen 3 applications, as the main intent here is to take destination address when flashing using DFU from the binary and not from specifications.js. Would you agree?

[busticated]

@avtolstoy if you think what we currently have is sufficient, that works for me 👍

do you need to release a new version of the CLI with these changes ASAP or can we just include them with the next update whenever that comes?

[avtolstoy]

@busticated Ideally we'd need it released before 3.1.0-rc.1, as DFU flashing of user applications will stop working for Gen 3 platforms, but since everything else is going to work (flashing over serial or OTA and Workbench is not affected), probably not that urgent. We are also dependent on cloud-side deploy, so nothing is going out until around June 3rd.

[busticated]

ok, sounds good - i'll merge it 👍