Merge pull request #777 from SignalK/doc_updates

Introduce v3 concepts in the documentation

README.md

@@ -45,9 +45,6 @@
   - `Throttle`: limits the rate of output updates
   - `Filter`: emits the input value only if it passes a given test
 
-- Support asynchronous configurables for setting remote device configuration
-  using the web interface
-
 - Implement stream producers that emit characters or lines from a stream
   (e.g., a serial port)
 - SKEmitter::as_signalk() has been renamed to SKEmitter::as_signalk_json().
@@ -119,9 +116,6 @@
 
 - Remove the final `sensesp_app->start();` call from your `setup()` function.
 
-- If your project depends on any external libraries, they may need to be updated
-  to work with the new version of SensESP.
-
 - The `reactesp` namespace is no longer imported. If you have any references to
   classes in this namespace, you will need to update them to use the namespace
   explicitly.

docs/index.md

@@ -19,14 +19,14 @@ SensESP features include:
 
 * High-level programming interfaces for sensor development
 * Support for a wide range of common sensor hardware with a set of [add-on libraries](pages/additional_resources/) - and if native support is missing, using existing Arduino libraries directly is also quite simple in most cases
-* A Web configuration user interface for sensors, transforms, and output paths
+* A Web user interface for configuring all aspects of the sensor system
 * Easy on-boarding with a Wi-Fi configuration tool and fully automated server discovery
 * Full Signal K integration with authentication, and transmission and reception of data
 * Support for over-the-air (OTA) firmware updates
-* Support for remote debugging over Wi-Fi
+
 
 To use SensESP, you need an ESP32 development board and a way to power it from the boat's 12V or 24V nominal power system.
-This can be done with commonly available ESP32DevKit boards and external DC-DC converters, or alternatively, the [Sailor Hat with ESP32 (SH-ESP32)](https://hatlabs.fi/product/sailor-hat-with-esp32/) has all these features baked into a developer-friendly board and enclosure kit.
+A great option, that will also support SensESP development is any of the [boards offered by Hat Labs](https://shop.hatlabs.fi/collections/sh-esp32-devices). However, any ESP32 board will work fine.
 
 Example use cases of SensESP include:
 

docs/pages/concepts/index.md

@@ -10,7 +10,7 @@ nav_order: 50
 
 ### Basics
 
-[ReactESP](https://github.com/mairas/ReactESP) is an asynchronous programming library for the ESP32 platform.
+[ReactESP](https://github.com/mairas/ReactESP) is an asynchronous programming and event loop library for the ESP32 platform.
 SensESP builds on top of ReactESP and uses it extensively under the hood.
 If you want to build more complex programs on top of SensESP or want to hack on SensESP internals, some basic understanding on the ReactESP basic concepts is highly useful.
 
@@ -37,7 +37,7 @@ void setup() {
     pinMode(kGpioPin, OUTPUT);
 
     // create the repeat event
-    app.onRepeat(interval, toggle_gpio);
+    event_loop()->onRepeat(interval, toggle_gpio);
 
     // setup continues
     ...
@@ -56,9 +56,9 @@ Another commonly used and useful time-based event is `DelayEvent`.
 It triggers after a certain amount of time has passed but does not repeat.
 Example use cases for that would be sensor devices in which you trigger the read operation and then come back to get the value after a certain amount of time.
 For example, the 1-Wire DS18B20 sensor can take up to 750 ms before the conversion is ready.
-In that case, you would first trigger the call and then have something like `app.onDelay(750, read_sensor);` to come back later to read the value.
+In that case, you would first trigger the call and then have something like `event_loop()->onDelay(750, read_sensor);` to come back later to read the value.
 
-You can also use `app.onDelay(...)` with a zero delay to trigger the event as soon as possible, without blocking the main event loop.
+You can also use `event_loop()->onDelay(...)` with a zero delay to trigger the event as soon as possible, without blocking the main event loop.
 
 ### Lambdas
 
@@ -108,7 +108,7 @@ It also supports the following:
 
 ### Removing Events
 
-All of the `app.onXXX()` calls return a `Event` object.
+All of the `event_loop()->onXXX()` calls return a `Event` object.
 If this object is stored, it can be used to access and manipulate the event later.
 In practice, you can disable the event by calling `event->remove()`.
 The same event can be re-added later by calling `event->add()`.
@@ -160,13 +160,11 @@ You can also include multiple Sensors, each with at least one Transform, in the
 
 ## Configuration Paths
 
-Some Sensors and Transform have parameters that can be configured at run-time. This section explains how to set them up. The user interface to change something at run-time is described [here](../user_interface/index.md). (BAS: this link doesn't work.)
+Configuration paths allow objects to persist their configuration to the device file system. Having a config_path defined also is a requirement for exposing the object to the web interface.
 
-In every case of a configurable value, its value can be set in `main.cpp` with a parameter to the constructor of the Sensor or Transform. By making it configurable, it's easier to make adjustments to your output based on what you're seeing in the real world. For example, the Median Transform is used to smooth output from a "noisy" sensor. By making the "Sample size" configurable, you can experiment with different sample sizes while the MCU is running and outputting data to Signal K, so you can decide when you have the right sample size for your purposes.
+Many Sensors and Transforms have parameters that can be configured at run-time. This section explains how to set them up.
 
-Look at the three lines from the `rpm_counter.cpp` example above. There are three constructors - one for the DigitalInputCounter Sensor, one for the Frequency Transform, and one for the SKOutputFloat Transform. The latter two have a configuration path included as the last item in their parameter list, but the first one doesn't. That means that the first one - the DigitalInputCounter - has no values that can be configured "live", but the latter two do. Actually, it's a little more complicated than that, because it's not entirely consistent among all Sensors and Transforms, and it's always optional whether you enable the ability to do the configuration.
-
-Even though a Sensor or Transform has the *ability* to be configurable, it won't *be* configurable unless you provide a configuration path in the constructor when you use it in `main.cpp`. For example,
+Even though a Sensor or Transform has the *ability* to be configurable, it won't *be* configurable unless you provide a configuration path in the constructor when you use it in `main.cpp` and call `ConfigItem` on it. For example,
 
 ```c++
 auto* analog_input = new AnalogInput();
@@ -180,44 +178,31 @@ auto* analog_input = new AnalogInput(250);
 
 creates a Sensor with a 250 ms Read Delay that still can't be adjusted in real time, because there is no config_path parameter.
 
-But
+Defining a config_path parameter allows the object to save its configuration to the device file system:
 
 ```c++
 auto* analog_input = new AnalogInput(250, "/analogInput");
 ```
 
-creates a Sensor with a 250 ms Read Delay that *can* be adjusted in real time, because of the presence of the config_path parameter (`"/analogInput"`).
-
-Your configuration path parameter can be passed with a variable you create, like this:
+Your configuration path parameter can also be passed with a variable you create, like this:
 
 ```c++
 const char* sensor_config_path = "/analogInput";
 auto* analog_input = new AnalogInput(250, sensor_config_path);
 ```
 
-or by putting the configuration path string directly into the parameter list of the constructor, like this:
+Now, if you want to expose the object to the web interface, call `ConfigItem` on it:
 
 ```c++
-auto* analog_input = new AnalogInput(250, "/analogInput");
+ConfigItem(analog_input)
+  ->set_title("Analogue Input")
+  ->set_description("Analog input read interval adjustment.")
+  ->set_sort_order(1100);
 ```
 
-### Naming Configuration Paths
-
-The naming of the paths is important, especially when you have multiple Sensors and / or multiple Transforms in your Project, so to be safe, please follow these guidelines:
-
-- Every configuration path name MUST begin with a forward slash.
-- Use two levels in your names, so that they look like `"/firstLevel/secondLevel"`, with the first level being a word that groups entries together in a logical manner, and the second level referring to the specific Sensor or Transform that the configuration path relates to. For example:
-
-Two Sensors (one for black water and one for fresh water), each using a Moving Average Transform and outputting to the Signal K Server with SKOutputFloat:
-
-- "/blackWater/analogInput" (for the blackwater AnalogInput() constructor in `main.cpp`)
-- "/blackWater/movingAvg" (for the blackwater MovingAverage() constructor in `main.cpp`)
-- "/blackWater/skPath" (for the blackwater SKOutputFloat() constructor in `main.cpp`)
-- "/freshWater/analogInput" (for the fresh water AnalogInput() constructor)
-- "/freshWater/movingAvg" (for the fresh water MovingAverage() constructor)
-- "/freshWater/skPath" (for the fresh water SKOutputFloat() constructor)
+That adds the created `analog_input object` to the web UI.
 
-This will group the configuration entries in the web interface into two groups: "blackWater" and "freshWater". Each group will have three entries: "analogInput", "movingAvg", and "SKOutput". Each "analogInput" entry will have one configurable value: "Read delay"; each "movingAvg" entry will have two configurable values: "Number of samples" and "Multiplier"; and each "skPath" entry will have one configurable value: "SignalK Path".
+The config_path is used only for saving the configuration to the device file system. The paths must be unique and start with a forward slash.
 
 ## Signal K Paths
 

docs/pages/getting_started/index.md

@@ -14,15 +14,10 @@ If you are a total newbie to software development and microcontrollers, see the
 ## Prerequisites: Hardware
 
 To get started with SensESP development, first you need some suitable hardware.
-A good low-cost option is an ESP32-DevKit board.
-Search AliExpress, Amazon, Ebay, or other marketplace of your choice for "ESP32-DevKitC v4".
-Pick one with a WROOM-32D or WROOM-32E module.
-Others are available and will likely work, but those are the sure-fire ones.
+The [SH-ESP32](https://shop.hatlabs.fi/products/sh-esp32) or [HALMET](https://shop.hatlabs.fi/products/halmet) boards by Hat Labs are great options, but generic
+ESP32 DevKit boards will work fine as well.
 
 The ESP32-DevKit boards are dirt cheap and great for testing and development on a lab desk, but integrating them in a "production" environment on a boat requires external power sources and enclosures and can be a hassle.
-For onboard purposes, a [Sailor Hat with ESP32 (SH-ESP32)](https://hatlabs.fi/product/sailor-hat-with-esp32/) is a great option.
-The SH-ESP32 has an integrated wide voltage range power supply, an NMEA-2000 compatible isolated CAN interface, a selection of suitable enclosures and connectors, and lots of other goodies for marine sensor development.
-And by purchasing one, you'll support SensESP development. ;-)
 
 While usable as is, you will get the most out of a SensESP device if you can connect it to a [Signal K](http://signalk.org/) server.
 Read the Signal K [Info](https://signalk.org/overview.html) and [Installation](https://signalk.org/installation.html) pages for more information on the hardware choices and setting one up.
@@ -79,14 +74,13 @@ You have to start the serial monitor manually by running `pio device monitor`.
 ## WiFi Configuration
 
 If a WiFi access point hasn't been configured yet, SensESP will create a special configuration access point for you.
-Open the list of nearby WiFi APs on your phone or computer and connect to the network named "Configure my-sensesp-project".
+Open the list of nearby WiFi APs on your phone or computer and connect to the network named "my-sensesp-project".
 The default network password is `thisisfine`.
 
 Normally, the WiFiManager configuration screen should automatically pop up.
 If that doesn't happen, browse to the captive portal IP address `192.168.4.1` in your web browser.
 
-Select the Wi-Fi network you prefer and enter the password, save and close.
-The device should now automatically connect to the network.
+An access point is enabled by default. To connect to an existing WiFi network, click on the WiFi Client checkbox and complete the configuration. Not that the access point will still remain enabled - if you want to disable it, uncheck the WiFi AP checkbox.
 
 Verify in the serial monitor output that the device connects properly to the WiFi.
 Assuming that is the case, you can now continue with the next step.

docs/pages/internals/index.md

@@ -66,56 +66,9 @@ By itself, it does nothing, but it implements the `ValueProducer` interface, and
 If you update the value of the `ObservableValue`, all the connected consumers will be notified.
 This approach can be used to inject arbitrary data into SensESP processing networks.
 
-## Configurables
+## Saveables and Serializables
 
-Many SensESP objects benefit from having a configuration interface and means for storing and retriveing configuration values from a persistent storage.
-The [`Configurable`](https://signalk.org/SensESP/generated/docs/classsensesp_1_1_configurable.html) class is a base class for all such objects.
-
-`Configurable` objects can read and write their configuration values by defining `set_configuration()` and `get_configuration()` methods.
-The method naming can be a bit confusing: `set_configuration()` sets the values of object member variables, while `get_configuration()` returns an `ArduinoJson` object filled with the member variable values.
-Hence, `set` loads and `get` saves the configuration.
-
-`Configurable` objects also normally define a config schema, acquired by calling `get_config_schema()`.
-The config schema is used to render the configuration page in the web interface.
-
-## Resettables and Startables
-
-Some additional base classes exist for objects that can be reset and that have additional startup routines.
-These inherit from [`Resettable`](https://signalk.org/SensESP/generated/docs/classsensesp_1_1_resettable.html) and [`Startable`](https://signalk.org/SensESP/generated/docs/classsensesp_1_1_startable.html) base classes.
-
-`Startable`s are more interesting and will be discussed first.
-
-A `Startable` object is something that should be somehow enabled or started when all other objects have been initialized and we want to actually start running the program.
-The startup routine is defined by the `start()` method implemented by the inheriting class.
-
-For example, basic WiFi networking objects should be initialized before the websocket connection to the Signal K server is attempted because the latter may utilize the former.
-It should be noted, however, that network connections, in particular, are established asynchronously.
-Returning from the `start()` call only signals that we have started establishing the connection, not that the connection already exists.
-
-The `Startable` constructor has an optional `priority` parameter that can be used to control the object startup order.
-Higher numbers come first and negative numbers are allowed.
-The priority definitions are arbitrary, but some of the currently defined values are:
-
-| Priority     | Subsystem            |
-|-------------:|:---------------------|
-| 80           | WiFi networking      |
-| 60           | SK Websocket client  |
-| 50           | HTTP server          |
-| 10           | Sensors              |
-| 5            | Transforms           |
-| 0            | Default value        |
-
-A `Resettable` object is something that should be called when the ESP device is factory reset.
-Currently, the only use cases for `Resettable` are initializing the file system and the WiFi network settings.
-
-Similar to `Startable`, the `Resettable` constructor has an optional `priority` parameter that can be used to control the object reset order.
-
-The currently defined values are:
-
-| Priority     | Subsystem            |
-|-------------:|:---------------------|
-| 0            | WiFi networking      |
-| -100         | File system          |
+SensESP can persist many objects to the local file system. In particular, class objects that inherit from `Serializable` can be serialized to JSON and deserialized back. Likewise, any class inheriting from 'FileSystemSaveable' allows saving and loading of the object to and from the file system.
 
 ## Sensors