Add rBoot partition support #2258

mikee47 · 2021-03-10T17:46:06Z

This PR adds partition table support to the ESP8266 bootloader (rBoot). See #2254 #2251.

The ESP8266 flash layout has been reverted to the original Sming 4.2 layout, with the addition of the partition table in the sector before the RF calibration data at end of flash.

Update rBoot

Update bootloader to read ROM addresses from partition table on boot. These overwrite whatever values have been set in its own configuration, thus avoiding inconsistencies. rBoot has been forked to simplify management.

An additional partition subtype has been added for RF calibration data, so existing applications running off the develop branch will need re-building and re-flashing.

The partition table may now be freely relocated for the Esp8266 (+ Host) to allow compatibility with existing devices which must be upgrade OTA.

Methods have been added to OTA classes to allow update regions to be set using partitions; this is safer than working with raw flash addresses but the underlying mechanism hasn't changed.

TODO:

Look at OTA support
Update documentation

mikee47 · 2021-03-13T22:37:38Z

OK, I've checked this out as follows on a nodeMCU with 4MByte flash memory:

Build and flash Basic_rBoot with Sming 4.2 LTS develop branch
Check it's all working - 'info' is a handy command
Check out this PR and go to the Basic_rBoot project directory.
Run:

make HWCONFIG=legacy/spiffs
make map

Check the map looks sensible. The partition table is at the end of flash; anywhere else and it could collide with SPIFFS, etc.

Flash the partition map, then read it back again to verify it hasn't been overwritten. (This happened for me when I put the partition table at 0x100000 - that's where SPIFFS is so got written over.)

make flashmap readmap

Finally, flash the new application:

make flashapp terminal

And we should now be running with Sming 4.3.

mikee47 · 2021-03-13T22:38:04Z

The OTA stuff I haven't yet tested. Volunteers?!

mikee47 · 2021-03-13T22:43:06Z

NB. This PR is on top of #2263 so I'll rebase once that's merged.

slaff · 2021-03-14T11:24:12Z

so I'll rebase once that's merged.

PR #2263 is merged. Feel free to rebase when you have time.

mikee47 · 2021-03-14T11:34:40Z

Minor fix to #2263 required...

Support ESP8266 partition table in any sector Support calculated `partition_table_offset` and partition `address` values

Sming/Components/Storage/src/include/Storage/PartitionTable.h

slaff · 2021-03-16T10:11:39Z

samples/Basic_Storage/basic_storage.hw

@@ -4,6 +4,7 @@
 	"devices": {
 		// Override default (conservative) flash settings for maximum performance
 		"spiFlash": {
+			// "mode": "qio", // This doesn't always work, depends on the flash chip


Replace the comment // This doesn't always work, depends on the flash chip with // Make sure to set the mode according to your flash chip. Allowed SPI modes are listed here: https://sming.readthedocs.io/en/latest/_inc/Sming/Components/esptool/index.html#envvar-SPI_MODE.

Sming/Components/rboot/include/Network/RbootHttpUpdater.h

slaff · 2021-03-16T10:23:19Z

Sming/Arch/Esp8266/options.json

@@ -6,5 +6,24 @@
                "filename": "$(FLASH_INIT_DATA_VCC)"
            }
        }
+    },
+    "legacy": {


@mikee47 Why we need a new memory layout for ESP8266 given the fact that now the partition table can be moved to a different location? Why not keep the current layout and not introduce a legacy layout?

OTA will function out of the box for ESP8266 applications

The firmware ROMs for big flashes can be as big as possible AND use the same entry point address

Doesn't work with regular paths or environment variables, has to be a URL. This is important because schemas can be integrated into IDE to ease editing of hardware profiles and IFS build configs.

slaff · 2021-03-16T16:52:36Z

Sming/Arch/Esp8266/standard.hw

-			"address": "0x008000",
-			"size": "992K",
+			"address": "0x002000",
+			"size": "0x0F8000",


@mikee47 Thanks a lot for the changes. Would it be possible to change the value here back to a human readable size in K? At least for me it is easier to see the max size without going to another tool to calculate the size in K.

FLASH_INIT_DATA is built as part of esp8266 Component, and written via partition mechanism There are no longer any FLASH_INIT_CHUNKS So flashinit -> wipe flash.

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

…2729) Leaving `blockErase` parameter at `false` when constructing a `PartitionStream` will result in corruption unless partition has already been erased. This is not always immediately apparent so can be difficult to diagnose. - **Breaking change** Replacing `blockErase` with an enumerated `mode` parameter enforces read-only behaviour by default (new feature and a good thing), with write access more explicitly defined. It's a simple change to existing code and will ensure it gets manually checked to ensure correct behaviour. - Update the `Basic_Ota` sample to check that an OTA update is not being attempted on the active running partition. This can happen with esp8266, for example, when running in temporary boot mode. (NB. Other checks may include active filing system partitions but that is application-specific.) - Update notes on migrating from Sming 4.2 to include updating boot sector. (Learning points froim #2727.) - Remove redundant flashsize calculation from rboot and update version string to `Sming v1.5` (from v1.4.2). Note this should have been done in #2258. - Fix rboot README regarding use of slot 2, should correspond with subtype `ota_2` not `ota_1`.

mikee47 force-pushed the feature/rboot-partition-support branch 2 times, most recently from 1deffb7 to a248a0c Compare March 10, 2021 18:41

slaff added this to the 4.3.0 milestone Mar 11, 2021

slaff mentioned this pull request Mar 11, 2021

OTA upgrade to Sming 4.3 #2251

Open

mikee47 force-pushed the feature/rboot-partition-support branch 8 times, most recently from 4d31392 to 787a900 Compare March 13, 2021 22:19

mikee47 changed the title ~~[WIP] Add rBoot partition support~~ Add rBoot partition support Mar 13, 2021

slaff added the 3 - Review label Mar 14, 2021

mikee47 force-pushed the feature/rboot-partition-support branch 2 times, most recently from 8761077 to 6932420 Compare March 14, 2021 11:27

mikee47 force-pushed the feature/rboot-partition-support branch 8 times, most recently from 62b57e5 to 736fb01 Compare March 15, 2021 13:01

Fork rboot with sming branch, add partition table support

cc06b73

Support ESP8266 partition table in any sector Support calculated `partition_table_offset` and partition `address` values

mikee47 added 4 commits March 16, 2021 07:25

Add Partition overload method

1a4a7c1

Tidy up

70d3818

Move addItem methods into header...

f46039c

Manage boot item list using RbootHttpUpdater::ItemList

a27f59e

mikee47 force-pushed the feature/rboot-partition-support branch from 736fb01 to a27f59e Compare March 16, 2021 07:25

slaff reviewed Mar 16, 2021

View reviewed changes

mikee47 added 2 commits March 16, 2021 16:22

Review changes

c1b1232

Fix vscode workspace schema paths

a78cdc4

Doesn't work with regular paths or environment variables, has to be a URL. This is important because schemas can be integrated into IDE to ease editing of hardware profiles and IFS build configs.

slaff reviewed Mar 16, 2021

View reviewed changes

mikee47 force-pushed the feature/rboot-partition-support branch 2 times, most recently from 7001a29 to decff1d Compare March 16, 2021 17:10

Get rid of 'legacy' options, just use those for defaults.

580dbd1

mikee47 force-pushed the feature/rboot-partition-support branch from decff1d to 580dbd1 Compare March 16, 2021 17:11

mikee47 added 5 commits March 16, 2021 20:02

Add buildpart target, add part-clean targets and run with clean

5b6ae6b

Add alternate option for Esp8266

dca2c2b

Remove redundant flashinit code

af73fe3

FLASH_INIT_DATA is built as part of esp8266 Component, and written via partition mechanism There are no longer any FLASH_INIT_CHUNKS So flashinit -> wipe flash.

Update migration notes

3fb8e58

Remove spiffs-image-update and spiffs-image-clean (now redundant)

7ade434

slaff removed the 3 - Review label Mar 17, 2021

slaff merged commit fec464b into SmingHub:develop Mar 17, 2021

slaff mentioned this pull request Mar 17, 2021

Preparation for Release 4.3.0 #2195

Closed

5 tasks

mikee47 deleted the feature/rboot-partition-support branch March 29, 2021 16:55

mikee47 pushed a commit to mikee47/Sming that referenced this pull request Mar 11, 2024

Update incorrect documentation regarding flash layout.

9d94a3c

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

mikee47 pushed a commit to mikee47/Sming that referenced this pull request Mar 12, 2024

Update incorrect documentation regarding flash layout.

700b528

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

mikee47 pushed a commit to mikee47/Sming that referenced this pull request Mar 12, 2024

Update incorrect documentation regarding flash layout.

4070188

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

mikee47 pushed a commit to mikee47/Sming that referenced this pull request Mar 12, 2024

Update incorrect documentation regarding flash layout.

95251b2

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

mikee47 pushed a commit to mikee47/Sming that referenced this pull request Mar 13, 2024

Update incorrect documentation regarding flash layout.

89124f7

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

mikee47 mentioned this pull request Mar 13, 2024

PartitionStream requires blockErase, other OTA-related improvements #2729

Merged

mikee47 pushed a commit to mikee47/Sming that referenced this pull request Mar 13, 2024

Update incorrect documentation regarding flash layout.

57cae57

Layout changed with the introduction of partition tables in PR SmingHub#2258, but reverted back to standard default layout in SmingHub#2332.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add rBoot partition support #2258

Add rBoot partition support #2258

mikee47 commented Mar 10, 2021 •

edited

Loading

mikee47 commented Mar 13, 2021

mikee47 commented Mar 13, 2021

mikee47 commented Mar 13, 2021

slaff commented Mar 14, 2021

mikee47 commented Mar 14, 2021

slaff Mar 16, 2021

slaff Mar 16, 2021

slaff Mar 16, 2021

Add rBoot partition support #2258

Add rBoot partition support #2258

Conversation

mikee47 commented Mar 10, 2021 • edited Loading

mikee47 commented Mar 13, 2021

mikee47 commented Mar 13, 2021

mikee47 commented Mar 13, 2021

slaff commented Mar 14, 2021

mikee47 commented Mar 14, 2021

slaff Mar 16, 2021

Choose a reason for hiding this comment

slaff Mar 16, 2021

Choose a reason for hiding this comment

slaff Mar 16, 2021

Choose a reason for hiding this comment

mikee47 commented Mar 10, 2021 •

edited

Loading