Skip to content

Jade OTA Hashes

Jump to bottom
Jamie C. Driver edited this page Sep 12, 2024 · 1 revision

Replying to: https://github.com/Blockstream/Jade/issues/160

Hello, I was wondering what the difference between cmphash and fwhash is: https://jadefw.blockstream.com/bin/jade1.1/index.json When I try to update my Jade from 0.1.41 to 1.0.31, my Jade seems to display the cmphash, which I believe is different from the hash that is generated following the instructions in REPRODUCIBLE.md (fwhash?). Projects like WalletScrutiny seem advertise the fwhash as well: https://walletscrutiny.com/hardware/blockstreamjade/#result
What is the difference between these two hashes? Are there any instructions on how to compute the cmphash?
Thank you.

Sorry, it gets a bit 'historical' ...

1.'cmphash' is the hash of the compressed fw file that is being uploaded to jade. It is trivially computed using sha256sum <file> (or in python using hashlib.sha256()). ie. it is the hash of the bytes we will send up to jade.
When we only had 'full' fw updates (ie the entire fw image in one file) this was ok, as the hash of the compressed file ~= the hash of the uncompressed file, ie the hash of the firmware image jade would boot/run.

Then we added ota 'deltas' - the diff between the currently running fw image, and the desired new/later fw. Since there could be many deltas, A->N, B->N, C->N etc. the hash of each (compressed) delta file would be different, even though the final fw image created is the same (and so would have the same hash).

So...
2. 'fwhash' is the hash of the full uncompressed firmware image that will actually run. It can be computed from the full firmware image file by decompressing it and then hashing:

cp <xxx_fw.bin file> tmp.fw.gz
pigz -z -d tmp.fw.gz
sha256sum tmp.fw

It is not possible to compute the 'fwhash' from a delta (xxx_patch.bin) file alone.

Both of these hashes should be present in the index.json file.

  • When you download a fw file you could check the hash matches the 'cmphash' given in the index ...
  • When you upload to jade, you can pass either/both of these hashes:
    • if passed 'cmphash' jade checks the bytes uploaded match that hash
    • if passed 'fwhash' jade checks the uncompressed (and possibly 'patched') final fw image bytes match that hash

NOTES:

  • the host/companion app needs to pass the hashes to Jade - any up-to-date app should be passing both, but an older app may predate adding 'fwhash' so may potentially only pass 'cmphash'
  • the running jade fw needs to have the logic to read and check those hashes if passed - checking 'cmphash' has been present from the outset, but verifying 'fwhash' was only added in fw 0.1.46 - This is why your 0.1.41 fw is only showing/checking 'cmphash'.

As above, checking the 'cmphash' yourself should be straighforward, and checking the corresponding 'fwhash' only slightly more onerous, requiring you to inflate the compressed fw file first.

Do some companions apps update using deltas, while others send the compressed file?

Probably, that's up to the app. I think the common strategy is 'prefer delta if one exists, otherwise use full fw file'. The idea was the deltas would be less to upload and hence faster to upgrade - but in effect most version upgrades are large enough that by the time you take into account the extra time/'difficulty' of patching, the overall ota time isn't much less with a delta. Saying that, if we did a small release (eg just one or two important bugfixes) the delta would probably be much faster to use.

NOTEs:

  1. the deltas are also compressed
  2. we only produce the deltas that are small enough to be worthwhile, eg. noradio <-> ble, vN-1 -> vN, maybe vN-2->vN. On particularly large releases we don't bother producing any deltas at all (ie. when they are all slower than using the full fw image file).

When updating using deltas, does the Jade “dry-run” apply them to its running firmware and calculate the fwhash to show to the user?

There is no "dry-run". In all cases (full or delta):

  1. the app passes the expected hashes to jade (as obtained from the index file) [*]

  2. jade shows the details to the user to confirm

  3. the fw is uploaded to jade, and jade verifies the hash was as initially stated (and confirmed by user) before finalising the new fw partition and flagging it as good.

  4. aside: the fw signatures are also checked before the partition is flagged, and indeed they are checked every time the current fw partition is booted.

[*] - the 'cmphash' is different for every file, as it's the hash of that (compressed) file. The 'fwhash' should be the same for every delta leading to a certain firmware - '1.0.30 ble' say, no matter what the starting point - as that is the hash of that final fw. That hash can't be extracted from a delta, but can be calculated from the relevant target full fw file - as explained in the previous post. ie. if the hash of uncompressed '1.0.30 ble' is X, then that should be the 'fwhash' of the full fw file for '1.0.30 ble' and for every delta file that leads to that fw, eg. '1.0.30 noradio -> 1.0.30 ble', '1.0.29 ble -> 1.0.30 ble', '1.0.28 ble -> 1.0.30 ble', etc.

NOTE: with a delta, the steps 'copy existing', 'apply delta', and 'update hash' happen as the delta file is being uploaded/decompressed.

ie. for full fw file:

  • User confirms values (incl hash) while bytes-to-upload:
    • upload compressed chunk of M bytes
    • update 'calculated cmphash' with M bytes
    • decompress to N bytes (expecting average N > M ofc)
    • update 'calculated fwhash' with N bytes
    • append N bytes to new fw partition
  • verify final 'calculated' hashes match those initially confirmed
  • check signatures
  • flag partition as good for next boot
  • reboot

and for delta:

  • User confirms values (incl hash) while bytes-to-upload:
    • upload compressed chunk of M bytes
    • update 'calculated cmphash' with M bytes
    • decompress to N bytes (expecting average N > M ofc)
    • read O bytes from current partition, applying any diffs [*]
    • update 'calculated fwhash' with P bytes [*]
    • append P bytes to new fw partition
  • verify final 'calculated' hashes match those initially confirmed
  • check signatures
  • flag partition as good for next boot
  • reboot

[*] the delta chunk of N bytes may, for example, contain info like, 'copy 1000 bytes unchanged, skip the next 100 source bytes, write these 250 bytes [<bytes>], copy the next 800 bytes unchanged ...' etc.
That's why when a delta is uploaded the progress can appear go in bursts - an instruction like 'copy the next 256kb unchanged' is a small upload but will look like a lot of new fw written, but 'then read 2kb, bytewise-add this 2kb to the original 2kb, and then write that 2kb' will be a lot slower.

NOTE: also the upload chunks are 4kb by default, but the decompression appears to run in ~12kb chunks - so we tend to see 3 or 4 'uploads' where we feed the bytes into the decompressor, but the the 'decompress' step results in no output ... then after the ~third chunk the 'decompress' step produces a ~32kb uncompressed output and a lot of writing happens.

Clone this wiki locally