From 0fc566d0f31f0a87261bae012593dc9afeb95bc9 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Mon, 30 Jul 2018 17:14:53 +0200 Subject: [PATCH 01/12] update "more flash" doc --- doc/faq/readme.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/faq/readme.rst b/doc/faq/readme.rst index e21f83b707..09e6da39ba 100644 --- a/doc/faq/readme.rst +++ b/doc/faq/readme.rst @@ -43,10 +43,12 @@ entering an issue report, please perform initial troubleshooting. How can I get some extra KBs in flash ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Using ``*printf()`` with floats is enabled by default. Some KBs of flash can -be saved by using the option ``--nofloat`` with the boards generator: +* Using ``*printf()`` with floats is enabled by default. Some KBs of flash can + be saved by using the option ``--nofloat`` with the boards generator: -``./tools/boards.txt.py --nofloat --allgen`` + ``./tools/boards.txt.py --nofloat --allgen`` + +* Use the debug level option ``NoAssert-NDEBUG`` (in the Tools menu) Why can't I use WPS ? ~~~~~~~~~~~~~~~~~~~~~ From abfd5bfb20c86f920131cf04faefa91de57aa457 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 00:49:24 +0200 Subject: [PATCH 02/12] doc: try to emphasize OOM knowledge --- doc/faq/a02-my-esp-crashes.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/faq/a02-my-esp-crashes.rst b/doc/faq/a02-my-esp-crashes.rst index c4e7753d6e..bcaf5f6021 100644 --- a/doc/faq/a02-my-esp-crashes.rst +++ b/doc/faq/a02-my-esp-crashes.rst @@ -10,7 +10,7 @@ My ESP crashes running some code. How to troubleshoot it? - `Exception <#exception>`__ - `Watchdog <#watchdog>`__ - `Check Where the Code Crashes <#check-where-the-code-crashes>`__ -- `Other Causes for Crashes <#other-causes-for-crashes>`__ +- `Other *Common* Causes for Crashes <#other-causes-for-crashes>`__ - `If at the Wall, Enter an Issue Report <#if-at-the-wall-enter-an-issue-report>`__ - `Conclusion <#conclusion>`__ @@ -263,7 +263,7 @@ Asynchronous Callbacks can happen. Memory, memory, memory - Running out of heap is the most common cause for crashes. Because the build process for + Running out of heap is the **most common cause for crashes**. Because the build process for the ESP leaves out exceptions (they use memory), memory allocations that fail will do so silently. A typical example is when setting or concatenating a large String. If allocation has failed internally, then the internal string copy can corrupt data, and @@ -286,6 +286,9 @@ Memory, memory, memory modification. For example, std::vector is available for use. The standard implementations rely on exceptions for error handling, which is not available for the ESP, and in any case there is no access to the underlying code. + + Instrumenting the code with the OOM debug option and calls to ``ESP.getFreeHeap()`` will + help the process of finding leaks. *Some techniques for reducing memory usage* From 9e6d9a0aecfe2a70c50cbdac6adf233186b7587f Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 00:52:52 +0200 Subject: [PATCH 03/12] fighting with markdown? --- doc/faq/a02-my-esp-crashes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/faq/a02-my-esp-crashes.rst b/doc/faq/a02-my-esp-crashes.rst index bcaf5f6021..1ee5353297 100644 --- a/doc/faq/a02-my-esp-crashes.rst +++ b/doc/faq/a02-my-esp-crashes.rst @@ -10,7 +10,7 @@ My ESP crashes running some code. How to troubleshoot it? - `Exception <#exception>`__ - `Watchdog <#watchdog>`__ - `Check Where the Code Crashes <#check-where-the-code-crashes>`__ -- `Other *Common* Causes for Crashes <#other-causes-for-crashes>`__ +- `Other **Common** Causes for Crashes <#other-causes-for-crashes>`__ - `If at the Wall, Enter an Issue Report <#if-at-the-wall-enter-an-issue-report>`__ - `Conclusion <#conclusion>`__ From aa57e7d3c0388bab08f93274fda2154952372e55 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 00:54:34 +0200 Subject: [PATCH 04/12] it seems I can't emphasize text in link --- doc/faq/a02-my-esp-crashes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/faq/a02-my-esp-crashes.rst b/doc/faq/a02-my-esp-crashes.rst index 1ee5353297..09ab9ebbef 100644 --- a/doc/faq/a02-my-esp-crashes.rst +++ b/doc/faq/a02-my-esp-crashes.rst @@ -10,7 +10,7 @@ My ESP crashes running some code. How to troubleshoot it? - `Exception <#exception>`__ - `Watchdog <#watchdog>`__ - `Check Where the Code Crashes <#check-where-the-code-crashes>`__ -- `Other **Common** Causes for Crashes <#other-causes-for-crashes>`__ +- `Other Common Causes for Crashes <#other-causes-for-crashes>`__ - `If at the Wall, Enter an Issue Report <#if-at-the-wall-enter-an-issue-report>`__ - `Conclusion <#conclusion>`__ From d0eb6b7f5202da00ff1bb5419fec6891ad3f57e4 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 01:52:21 +0200 Subject: [PATCH 05/12] board generator doc: need reviewers --- doc/faq/a05-board-generator.rst | 65 +++++++++++++++++++++++++++++++++ doc/faq/readme.rst | 14 +++++++ 2 files changed, 79 insertions(+) create mode 100644 doc/faq/a05-board-generator.rst diff --git a/doc/faq/a05-board-generator.rst b/doc/faq/a05-board-generator.rst new file mode 100644 index 0000000000..7878998022 --- /dev/null +++ b/doc/faq/a05-board-generator.rst @@ -0,0 +1,65 @@ +:orphan: + +Board generator +--------------- + +- `How can I run the script ? <#how-can-i-run-the-script>`__ +- `What can I do with it ? <#what-can-i-do-with-it>`__ +- `When do I need to update it ? <#when-do-i-need-to-mess-with-it>`__ +- `Why is my pull-request failing continuous-integration ? <#why-is-my-pull-request-failing-continuous-integration ?>`__ + +How can I run the script ? +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Python 2 needs to be installed on your system. +The script is located in the ``tools`` subdirectory of the core's root installation. +It needs to be run from the root directory, + +:: + + $ tools/boards.txt.py + +:: + + C:\...> tools\boards.txt.py + +Running without parameters will show the command line help. They are +generally self-explanatory. + + +What can I do with it ? +~~~~~~~~~~~~~~~~~~~~~~~ + +As of today you can +* in the IDE: change the default serial programming speed of any board +* in the IDE: add new serial programming speed +* increase available flash space by disabling floats in ``*printf`` functions +* enable WPS which is now disabled by default (at the cost of a smaller heap by 4KB) +* change led pin ``LED_BUILTIN`` for the two generic boards +* change the default lwIP version (1.4 or 2) + + +When do I need to mess with it ? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The board generator is used to automate configuration files when it is +possible to. It needs to be edited for: + +* All the specific board informations. This is the only place where a new + board (definition, description) can be updated or added to the existing + list. + +* Memory mapping for ldscripts (flash and spiffs size combinations) + + +Why are my pull-request failing continuous-integration ? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The generator is able to update a number of files (see list in help), and +global coherency can be checked by the continuous integration facilities. + +After a modification in the generator, it is **mandatory** to regenerate all +files (option ``--allgen``) and add them in the pull-request. + + +`FAQ list :back: `__ diff --git a/doc/faq/readme.rst b/doc/faq/readme.rst index 09e6da39ba..1885c49e25 100644 --- a/doc/faq/readme.rst +++ b/doc/faq/readme.rst @@ -141,3 +141,17 @@ The following lines are compatible with both lwIP versions: } Ref. `#1923 `__ + + +Why is there a board generator and what about it ? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The board generator is a python script originally intended to ease the +Arduino IDE's `boards.txt` configuration file about the multitude of +available board, especially when common data have to be updated for all of +them. + +This script is also used to manage uncommon options that will not find their +place into the IDE menu. + +`Read more `__. From 62526be63c2f9d760119ab901cb64c8065612690 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 01:57:08 +0200 Subject: [PATCH 06/12] markdown formatting, typos --- doc/faq/a05-board-generator.rst | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/doc/faq/a05-board-generator.rst b/doc/faq/a05-board-generator.rst index 7878998022..ecb0c0a8b7 100644 --- a/doc/faq/a05-board-generator.rst +++ b/doc/faq/a05-board-generator.rst @@ -12,6 +12,7 @@ How can I run the script ? ~~~~~~~~~~~~~~~~~~~~~~~~~~ Python 2 needs to be installed on your system. + The script is located in the ``tools`` subdirectory of the core's root installation. It needs to be run from the root directory, @@ -30,19 +31,25 @@ generally self-explanatory. What can I do with it ? ~~~~~~~~~~~~~~~~~~~~~~~ -As of today you can +As of today you can: + * in the IDE: change the default serial programming speed of any board + * in the IDE: add new serial programming speed + * increase available flash space by disabling floats in ``*printf`` functions + * enable WPS which is now disabled by default (at the cost of a smaller heap by 4KB) + * change led pin ``LED_BUILTIN`` for the two generic boards + * change the default lwIP version (1.4 or 2) When do I need to mess with it ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The board generator is used to automate configuration files when it is +The board generator is used to automate configuration files generatiom when it is possible to. It needs to be edited for: * All the specific board informations. This is the only place where a new @@ -52,7 +59,7 @@ possible to. It needs to be edited for: * Memory mapping for ldscripts (flash and spiffs size combinations) -Why are my pull-request failing continuous-integration ? +Why is my pull-request failing continuous-integration ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The generator is able to update a number of files (see list in help), and From 9ac20a160d66294aaca4059d887d17e333313913 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 02:00:16 +0200 Subject: [PATCH 07/12] typos --- doc/faq/a05-board-generator.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/faq/a05-board-generator.rst b/doc/faq/a05-board-generator.rst index ecb0c0a8b7..3a641354c1 100644 --- a/doc/faq/a05-board-generator.rst +++ b/doc/faq/a05-board-generator.rst @@ -6,7 +6,7 @@ Board generator - `How can I run the script ? <#how-can-i-run-the-script>`__ - `What can I do with it ? <#what-can-i-do-with-it>`__ - `When do I need to update it ? <#when-do-i-need-to-mess-with-it>`__ -- `Why is my pull-request failing continuous-integration ? <#why-is-my-pull-request-failing-continuous-integration ?>`__ +- `Why is my pull-request failing continuous-integration ? <#why-is-my-pull-request-failing-continuous-integration>`__ How can I run the script ? ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -39,7 +39,7 @@ As of today you can: * increase available flash space by disabling floats in ``*printf`` functions -* enable WPS which is now disabled by default (at the cost of a smaller heap by 4KB) +* enable WPS which is now disabled by default (at the cost of a smaller heap by ~4KB) * change led pin ``LED_BUILTIN`` for the two generic boards @@ -49,7 +49,7 @@ As of today you can: When do I need to mess with it ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The board generator is used to automate configuration files generatiom when it is +The board generator is used to automate configuration files generation when it is possible to. It needs to be edited for: * All the specific board informations. This is the only place where a new @@ -60,7 +60,7 @@ possible to. It needs to be edited for: Why is my pull-request failing continuous-integration ? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The generator is able to update a number of files (see list in help), and global coherency can be checked by the continuous integration facilities. From fabd7b37aebcdc75ffcee45085f90c86d74a11ed Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 02:09:02 +0200 Subject: [PATCH 08/12] add links --- doc/faq/readme.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/faq/readme.rst b/doc/faq/readme.rst index 1885c49e25..a53d068aae 100644 --- a/doc/faq/readme.rst +++ b/doc/faq/readme.rst @@ -50,6 +50,8 @@ How can I get some extra KBs in flash ? * Use the debug level option ``NoAssert-NDEBUG`` (in the Tools menu) +`Read more `__. + Why can't I use WPS ? ~~~~~~~~~~~~~~~~~~~~~ @@ -58,6 +60,8 @@ WPS (and lose 4KB of useable ram), use this boards generator option: ``./tools/boards.txt.py --allowWPS --allgen`` +`Read more `__. + This Arduino library doesn't work on ESP. How do I make it work? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 10982ef10c34044e6c6306fe987d85c13c605756 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 11:56:25 +0200 Subject: [PATCH 09/12] + last memory failure allocation + emphasis exception decoder --- doc/faq/a02-my-esp-crashes.rst | 28 +++++++++++++++++----------- doc/faq/a05-board-generator.rst | 4 ++-- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/doc/faq/a02-my-esp-crashes.rst b/doc/faq/a02-my-esp-crashes.rst index 09ab9ebbef..f56b3ce79f 100644 --- a/doc/faq/a02-my-esp-crashes.rst +++ b/doc/faq/a02-my-esp-crashes.rst @@ -9,7 +9,7 @@ My ESP crashes running some code. How to troubleshoot it? - `What is the Cause of Restart? <#what-is-the-cause-of-restart>`__ - `Exception <#exception>`__ - `Watchdog <#watchdog>`__ -- `Check Where the Code Crashes <#check-where-the-code-crashes>`__ +- `Exception Decoder <#exception-decoder>`__ - `Other Common Causes for Crashes <#other-causes-for-crashes>`__ - `If at the Wall, Enter an Issue Report <#if-at-the-wall-enter-an-issue-report>`__ @@ -161,8 +161,8 @@ out before restart, you should be able to narrow down part of code firing the h/w wdt reset. If diagnosed application or library has debug option then switch it on to aid this troubleshooting. -Check Where the Code Crashes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Exception Decoder +~~~~~~~~~~~~~~~~~ Decoding of ESP stack trace is now easy and available to everybody thanks to great `Arduino ESP8266/ESP32 Exception Stack Trace @@ -183,8 +183,10 @@ If you don't have any code for troubleshooting, use the example below: Serial.println(); Serial.println("Let's provoke the s/w wdt firing..."); // - // wait for s/w wdt in infinite loop below + // provoke an OOM, will be recorded as the last occured one + char* out_of_memory_failure = (char*)malloc(1000000); // + // wait for s/w wdt in infinite loop below while(true); // Serial.println("This line will not ever print out"); @@ -192,12 +194,14 @@ If you don't have any code for troubleshooting, use the example below: void loop(){} -Upload this code to your ESP (Ctrl+U) and start Serial Monitor -(Ctrl+Shift+M). You should shortly see ESP restarting every couple of -seconds and ``Soft WDT reset`` message together with stack trace showing -up on each restart. Click the Autoscroll check-box on Serial Monitor to -stop the messages scrolling up. Select and copy the stack trace, go to -the *Tools* and open the *ESP Exception Decoder*. +Enable the Out-Of-Memory (*OOM*) debug option (in the *Tools > Debug Level* +menu), compile/flash/upload this code to your ESP (Ctrl+U) and start Serial +Monitor (Ctrl+Shift+M). You should shortly see ESP restarting every couple +of seconds and ``Soft WDT reset`` message together with stack trace showing +up on each restart. Click the Autoscroll check-box on Serial Monitor to +stop the messages scrolling up. Select and copy the stack trace, including +the ``last failed alloc call: ...`` line, go to the *Tools* and open the +*ESP Exception Decoder*. .. figure:: pictures/a02-decode-stack-tace-1-2.png :alt: Decode the stack trace, steps 1 and 2 @@ -286,10 +290,12 @@ Memory, memory, memory modification. For example, std::vector is available for use. The standard implementations rely on exceptions for error handling, which is not available for the ESP, and in any case there is no access to the underlying code. - + Instrumenting the code with the OOM debug option and calls to ``ESP.getFreeHeap()`` will help the process of finding leaks. + + *Some techniques for reducing memory usage* * Don't use const char * with literals. Instead, use const char[] PROGMEM. This is particularly true if you intend to, e.g.: embed html strings. diff --git a/doc/faq/a05-board-generator.rst b/doc/faq/a05-board-generator.rst index 3a641354c1..e5863d1fe7 100644 --- a/doc/faq/a05-board-generator.rst +++ b/doc/faq/a05-board-generator.rst @@ -16,7 +16,7 @@ Python 2 needs to be installed on your system. The script is located in the ``tools`` subdirectory of the core's root installation. It needs to be run from the root directory, -:: +:: $ tools/boards.txt.py @@ -55,7 +55,7 @@ possible to. It needs to be edited for: * All the specific board informations. This is the only place where a new board (definition, description) can be updated or added to the existing list. - + * Memory mapping for ldscripts (flash and spiffs size combinations) From 832d79d167529d765cd6fa2891c1622c5d3ec414 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 12:00:48 +0200 Subject: [PATCH 10/12] more emphasis on exception decoder --- doc/faq/a02-my-esp-crashes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/faq/a02-my-esp-crashes.rst b/doc/faq/a02-my-esp-crashes.rst index f56b3ce79f..cb60e6b32f 100644 --- a/doc/faq/a02-my-esp-crashes.rst +++ b/doc/faq/a02-my-esp-crashes.rst @@ -292,8 +292,8 @@ Memory, memory, memory case there is no access to the underlying code. Instrumenting the code with the OOM debug option and calls to ``ESP.getFreeHeap()`` will - help the process of finding leaks. - + help the process of finding leaks. Now is time to re-read about the + `exception decoder <#exception-decoder>`__. *Some techniques for reducing memory usage* From 868b17a7e9123431b2bf60b1e40ab3cad88000b4 Mon Sep 17 00:00:00 2001 From: David Gauchard Date: Tue, 31 Jul 2018 12:25:40 +0200 Subject: [PATCH 11/12] repeat the board generator introduction --- doc/faq/a05-board-generator.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/doc/faq/a05-board-generator.rst b/doc/faq/a05-board-generator.rst index e5863d1fe7..de8df5a486 100644 --- a/doc/faq/a05-board-generator.rst +++ b/doc/faq/a05-board-generator.rst @@ -3,6 +3,14 @@ Board generator --------------- +The board generator is a python script originally intended to ease the +Arduino IDE's `boards.txt` configuration file about the multitude of +available board, especially when common data have to be updated for all of +them. + +This script is also used to manage uncommon options that will not find their +place into the IDE menu. + - `How can I run the script ? <#how-can-i-run-the-script>`__ - `What can I do with it ? <#what-can-i-do-with-it>`__ - `When do I need to update it ? <#when-do-i-need-to-mess-with-it>`__ From 698b0813c543523ba64127cf2b0f527d03b6c93f Mon Sep 17 00:00:00 2001 From: david gauchard Date: Tue, 31 Jul 2018 23:12:01 +0200 Subject: [PATCH 12/12] fixes --- doc/faq/a05-board-generator.rst | 15 ++++++++------- doc/faq/readme.rst | 8 ++++---- 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/doc/faq/a05-board-generator.rst b/doc/faq/a05-board-generator.rst index de8df5a486..03ad412f91 100644 --- a/doc/faq/a05-board-generator.rst +++ b/doc/faq/a05-board-generator.rst @@ -5,11 +5,11 @@ Board generator The board generator is a python script originally intended to ease the Arduino IDE's `boards.txt` configuration file about the multitude of -available board, especially when common data have to be updated for all of -them. +available boards, especially when common parameters have to be updated for +all of them. -This script is also used to manage uncommon options that will not find their -place into the IDE menu. +This script is also used to manage uncommon options that are currently not +available in the IDE menu. - `How can I run the script ? <#how-can-i-run-the-script>`__ - `What can I do with it ? <#what-can-i-do-with-it>`__ @@ -31,6 +31,7 @@ It needs to be run from the root directory, :: C:\...> tools\boards.txt.py + C:\...> python tools\boards.txt.py Running without parameters will show the command line help. They are generally self-explanatory. @@ -57,10 +58,10 @@ As of today you can: When do I need to mess with it ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The board generator is used to automate configuration files generation when it is -possible to. It needs to be edited for: +The board generator is used to automate generation of configuration files +when possible. It needs to be edited for: -* All the specific board informations. This is the only place where a new +* All information for specific boards. This is the only place where a new board (definition, description) can be updated or added to the existing list. diff --git a/doc/faq/readme.rst b/doc/faq/readme.rst index a53d068aae..2ed3743a50 100644 --- a/doc/faq/readme.rst +++ b/doc/faq/readme.rst @@ -152,10 +152,10 @@ Why is there a board generator and what about it ? The board generator is a python script originally intended to ease the Arduino IDE's `boards.txt` configuration file about the multitude of -available board, especially when common data have to be updated for all of -them. +available boards, especially when common parameters have to be updated for +all of them. -This script is also used to manage uncommon options that will not find their -place into the IDE menu. +This script is also used to manage uncommon options that are currently not +available in the IDE menu. `Read more `__.