Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for IDF 4.4 release #710

Closed
Jason2866 opened this issue Jan 27, 2022 · 37 comments
Closed

Support for IDF 4.4 release #710

Jason2866 opened this issue Jan 27, 2022 · 37 comments

Comments

@Jason2866
Copy link
Contributor

IDF v4.4 has been released by espressif. Arduino ESP32 2.0.2 is based on IDF 4.4

@FaBjE
Copy link

FaBjE commented Jan 27, 2022

I've created a pull-request based on the work of @X-Ryl669 and @jonathandreyer
here: #711 (Also includes a temp test configuration)

@Jason2866
Copy link
Contributor Author

@FaBjE Thx, based on your ground work i added the S3 and all CI examples (added tests for S2 and S3) do compile successfully.

@turing-complete-labs
Copy link

How can I test this PR with an S3 sample?
Also would it be worth having a ROADMAP.md document for people wondering when x.y IDF support will reach Platformio?

@Jason2866
Copy link
Contributor Author

Jason2866 commented Feb 3, 2022

@turing-complete-labs
platform = https://github.com/tasmota/platform-espressif32/releases/download/S3_alpha/platform-espressif32-S3-alpha.zip
The repo https://github.com/tasmota/platform-espressif32/tree/IDF44/ESP32-S3
examples are in example folder
The PR from FaBjE does not include/work for the S3

@quangvankts
Copy link

quangvankts commented Feb 9, 2022

@turing-complete-labs
platform = https://github.com/tasmota/platform-espressif32/releases/download/S3_alpha/platform-espressif32-S3-alpha.zip
The repo https://github.com/tasmota/platform-espressif32/tree/IDF44/ESP32-S3
examples are in example folder
The PR from FaBjE does not include/work for the S3

@Jason2866
I'm having error compiling for my ESP32-S3 example
My platformio.ini

[env:esp32-s3]
platform = https://github.com/tasmota/platform-espressif32/releases/download/S3_alpha/platform-espressif32-S3-alpha.zip
board = esp32-s3-devkitc-1
framework = arduino
board_build.f_flash = 80000000L
monitor_speed = 115200

Output:

Linking .pio/build/esp32-s3/firmware.elf
xtensa-esp32s3-elf-g++: error: S3.map: No such file or directory
*** [.pio/build/esp32-s3/firmware.elf] Error 1

@Jason2866
Copy link
Contributor Author

Jason2866 commented Feb 9, 2022

@quangvankts Try to delete hidden folder .platformio in your user path.
Platformio will rebuild and fetch all needed. Probably you have stuff left which concflicts.
This board_build.f_flash = 80000000L is not needed since it is in boards.json already
There is a S3 Blink example you can use https://github.com/tasmota/platform-espressif32/tree/IDF44/ESP32-S3/examples/arduino-rmt-blink
Successfull Github Actions CI run https://github.com/Jason2866/platform-espressif32/runs/5108450603?check_suite_focus=true

@rbray89
Copy link

rbray89 commented Feb 9, 2022

As a first-step, can the 4.4 package be uploaded to https://registry.platformio.org/tools/platformio/framework-espidf ? The ESPHome project uses the platformio registry, and there are new features in the 4.4 version I need access to.

@quangvankts
Copy link

quangvankts commented Feb 10, 2022

@quangvankts Try to delete hidden folder .platformio in your user path.
Platformio will rebuild and fetch all needed. Probably you have stuff left which concflicts.
This board_build.f_flash = 80000000L is not needed since it is in boards.json already
There is a S3 Blink example you can use https://github.com/tasmota/platform-espressif32/tree/IDF44/ESP32-S3/examples/arduino-rmt-blink
Successfull Github Actions CI run https://github.com/Jason2866/platform-espressif32/runs/5108450603?check_suite_focus=true

I deleted hidden folder user/.platformio, and waited for PIO to rebuild, but the exact same error returned.

However changing the platformio.ini back to

platform = espressif32
board = esp32dev

I can compile with no error

@turing-complete-labs
Copy link

@quangvankts I had the same issue, since I created the project from scratch.
Reusing an example (rtm-arduino-blink) did the trick although not without issues.

@aovestdipaperino
Copy link

Espressif 3.5.0 is out and it supports S3.

@X-Ryl669
Copy link

X-Ryl669 commented Feb 20, 2022

No it doesn't. Espressif 3.5 is based on IDF4.3 which does not support ESP32 S3 (it supports ESP32 C3 however).
Support for S3 requires ESP-IDF 4.4 at least.

@mrengineer7777
Copy link

Linking .pio/build/esp32-s3/firmware.elf
xtensa-esp32s3-elf-g++: error: S3.map: No such file or directory
*** [.pio/build/esp32-s3/firmware.elf] Error 1

I've seen this issue before. The map file should match the name of your project. Make sure there are no spaces in your project name. Also verify that your project path doesn't have any spaces in it. Clean and rebuild.

@mrengineer7777
Copy link

+1 for IDF 4.4 support and Arduino 2.0.2. Long overdue. Fortunately we can use TASMOTA custom builds as a workaround.

@Jason2866
Copy link
Contributor Author

Jason2866 commented Mar 11, 2022

Arduino Core 2.0.3 is nearly there. Here is my core 2.0.3 rc build from today

platform = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.2.3/platform-espressif32-2.0.2.3.zip
platform_packages = framework-arduinoespressif32 @ https://github.com/Jason2866/esp32-arduino-lib-builder/releases/download/728/framework-arduinoespressif32-v4.4_dev-d3c108c910.tar.gz

When core 2.0.3 is released i will do a "clean" platform build and update IDF44 to the one which is used too build the Arduino libs.

@CSharperMantle
Copy link

CSharperMantle commented Mar 21, 2022

+1 for latest ESP-IDF support. I found incredibly ancient APIs (e.g., tcpip_adapter_get_ip_info and the header tcpip_adapter.h, which are not even found in docs and deprecated by ESP-NETIF interfaces) used in current WiFi library while the official espressif/arduino-esp32 has moved on far forward.

Also, can framework-arduinoespressif32 be updated at the same time?

@Jason2866
Copy link
Contributor Author

@CSharperMantle Latest IDF44 and Arduino core 2.0.3-rc1 support.

platform = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.3rc1/platform-espressif32-2.0.3.zip

Lets hope the devs of PlatformIO in Ukraine are safe and have soon time (when war is over) for there business again.

@TheButlah
Copy link

TheButlah commented Apr 23, 2022

Doesn't the current version of platform-espressif32 (4.1.0) that was released 2 days ago use framework-arduinoespressif32 v3.20001.0, which uses arduino-esp32 v2.0.1, which uses ESP-IDF v4.4?

So this issue is already addressed by the latest platform release I think, please correct me if I'm wrong as I am super unfamiliar with how the platform's dependencies work.

@Jason2866
Copy link
Contributor Author

No, it is still IDF4.3 to be exact IDF4.3.2 as referenced here https://github.com/platformio/platform-espressif32/blob/develop/platform.json#L81

@jonathandreyer
Copy link

Hello everyone,

I write a quick message here to inform you that I have updated the version for ESP-IDF v4.4.1.
In this version it is only necessary to replace the platform reference by:

platform = https://github.com/jonathandreyer/platform-espressif32.git#espidf-441

I have already tested on two targets ESP32 & ESP32-S3.

Best regards,

Jonathan Dreyer

@Jason2866
Copy link
Contributor Author

Hello @jonathandreyer so we are too with the same idea ;-)

platform = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.3rc1/platform-espressif32-2.0.3new.zip

@TheButlah
Copy link

No, it is still IDF4.3 to be exact IDF4.3.2 as referenced here https://github.com/platformio/platform-espressif32/blob/develop/platform.json#L81

Oh was this issue for framework-espidf? I had assumed it was about framework-arduinoespressif32

@johnboiles
Copy link

johnboiles commented Apr 26, 2022

@Jason2866 I've been using the tasmota 4.4 build for a while, but this new 4.4.1 one you just posted 2 days ago fails like this:

TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType:
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/platformio/builder/main.py", line 181:
    env.SConscript("$BUILD_SCRIPT")
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/Users/johnboiles/.platformio/platforms/espressif32/builder/main.py", line 282:
    target_elf = env.BuildProgram()
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Util.py", line 742:
    return self.method(*nargs, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/platformio/builder/tools/platformio.py", line 61:
    env.ProcessProgramDeps()
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Util.py", line 742:
    return self.method(*nargs, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/platformio/builder/tools/platformio.py", line 128:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Util.py", line 742:
    return self.method(*nargs, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/platformio/builder/tools/platformio.py", line 343:
    SConscript(env.GetFrameworkScript(f), exports="env")
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 660:
    return method(*args, **kw)
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/Users/johnboiles/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/Users/johnboiles/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 63:
    assert os.path.isdir(TOOLCHAIN_DIR)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/genericpath.py", line 42:
    st = os.stat(s)

I've tried rm -rf .pio && rm -rf ~/.platformio.

@Jason2866
Copy link
Contributor Author

Mhh, do you have sample program where i can try?
The CI examples (Arduino IDF) are all successfull.
https://github.com/Jason2866/platform-espressif32/actions/runs/2216174461

@bwjohns4
Copy link

@jonathandreyer, do you know what I need to do for this to compile on a Mac? I get this error:
Error: Could not find the package with 'jonathandreyer/framework-espidf @ ~3.40001.0' requirements for your system 'darwin_arm64'

@Jason2866
Copy link
Contributor Author

@bwjohns4 Use my fork (i added my compiled native Apple silicon toolchains)
Just change this

platform  = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.3rc1/platform-espressif32-2.0.3new.zip

@bwjohns4
Copy link

bwjohns4 commented May 1, 2022

@Jason2866 , trying yours I get a different error (shown below). Any insight on what I'm doing wrong? I'm trying to build with Arduino and IDF from scratch.

[env:esp32devIDF2]
platform = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.3rc1/platform-espressif32-2.0.3new.zip
framework = arduino, espidf
board = esp32dev

PACKAGES:

  • framework-arduinoespressif32 2.0.3
  • framework-espidf 3.40401.0 (4.4.1)
  • tool-cmake 3.16.4
  • tool-esptoolpy 1.30300.220323 (3.3.0)
  • tool-mklittlefs 1.203.210628 (2.3)
  • tool-mkspiffs 2.230.0 (2.30)
  • tool-ninja 1.9.0
  • toolchain-esp32ulp 1.22851.191205 (2.28.51)
  • toolchain-xtensa-esp32-arm 8.4.0+2021r2-patch3
    TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType:
    File "/Users/bwjohns4/.platformio/penv/lib/python3.8/site-packages/platformio/builder/main.py", line 181:
    env.SConscript("$BUILD_SCRIPT")
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
    File "/Users/bwjohns4/.platformio/platforms/espressif32/builder/main.py", line 282:
    target_elf = env.BuildProgram()
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Util.py", line 742:
    return self.method(*nargs, **kwargs)
    File "/Users/bwjohns4/.platformio/penv/lib/python3.8/site-packages/platformio/builder/tools/platformio.py", line 61:
    env.ProcessProgramDeps()
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Util.py", line 742:
    return self.method(*nargs, **kwargs)
    File "/Users/bwjohns4/.platformio/penv/lib/python3.8/site-packages/platformio/builder/tools/platformio.py", line 128:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Util.py", line 742:
    return self.method(*nargs, **kwargs)
    File "/Users/bwjohns4/.platformio/penv/lib/python3.8/site-packages/platformio/builder/tools/platformio.py", line 343:
    SConscript(env.GetFrameworkScript(f), exports="env")
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 660:
    return method(*args, **kw)
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
    File "/Users/bwjohns4/.platformio/packages/tool-scons/scons-local-4.3.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
    File "/Users/bwjohns4/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 63:
    assert os.path.isdir(TOOLCHAIN_DIR)
    File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/genericpath.py", line 42:
    st = os.stat(s)

@Jason2866
Copy link
Contributor Author

Jason2866 commented May 1, 2022

Ahh, i did not test if the arm toolchains are used / working with IDF!
This one is tested with IDF (does not use native toolchains, so a bit slower)

[env:esp32devIDF2]
platform = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.3rc1/platform-espressif32-2.0.3.zip
framework = arduino, espidf
board = esp32dev

Thx, for the feedback. I try to solve the IDF issue with the other platform setup, so IDF will work with the native Apple silicon toolchains too.
If espressif is merging my PR for the native Apple toolchains this workaround is not needed anymore
@bwjohns4

@bwjohns4
Copy link

bwjohns4 commented May 1, 2022

@Jason2866 , Thanks so much for helping with this! It compiled! I've been trying to get this to work for weeks! I did notice that there were many compiler warnings that were treated as errors and failing the build (such as [-Werror=maybe-uninitialized]). I was able to find and use build_unflags = -Werror=all (https://community.platformio.org/t/disable-warning-errors/13280/2 apparently IDF has it flagged to treat all warnings as errors?) which solved the problem and it successfully built.

Most / all of the warnings that were failing the build were within my code and not within Arduino/IDF cores/libraries. Has anyone else seen this too? Should I not worry about it since for any regular Arduino build the code works fine the way it's currently written.

@Jason2866
Copy link
Contributor Author

Yes, IDF is compiling with -Werror=maybe-uninitialized which is IMHO a bit too much.
A warning should be enough.

@bwjohns4
Copy link

bwjohns4 commented May 2, 2022

@Jason2866 , I'm still wrestling a bit with this. It compiles but just does a repeating WDT reboot when uploaded to a device. I'm thinking that somehow I don't have sdkconfig right or something. Also when I turn on my debug serial prints, its fails to build due to being too large 104% of flash capacity versus ~82% capacity when building with the latest standard espressif32 arduino framework (the one that has pre-built IDF).

I've copied the sdkconfig file from https://github.com/espressif/arduino-esp32/blob/master/tools/sdk/esp32/sdkconfig into my sdkconfig.esp32devIDF2 file. Is there something different that I should be doing to ensure that IDF is built with all the correct flags and configuration that Arduino is using presently?

@Jason2866
Copy link
Contributor Author

Yes, there have settings changed in IDF4.4 for Arduino 2.0.x.
You have to look in Arduino framework (folder tools/skd/...)
Changes there are useless, it is the info! how the Arduino framework is compiled.
IDF and Arduino are compiled with the same sdkconfig settings

@Jason2866
Copy link
Contributor Author

Jason2866 commented May 4, 2022

@bwjohns4 The arm toolchains should now work for IDF too. Please try

platform = https://github.com/Jason2866/platform-espressif32/releases/download/v2.0.3-rc1/platform-espressif32-Tasmota-safe.zip

The Arduino framework is updated and used IDF441 for compiling

@liebman
Copy link

liebman commented Jun 4, 2022

@Jason2866
Using:

platform = https://github.com/tasmota/platform-espressif32/releases/download/v.2.0.3/platform-espressif32-v.2.0.3.zip

When running platformio device monitor I found that there is a : missing on this class definition:

  File "/Users/chris.l/.platformio/platforms/espressif32/monitor/filter_exception_decoder.py", line 32
    class Esp32ExceptionDecoder(DeviceMonitorFilterBase)
                                                        ^
SyntaxError: invalid syntax

@Jason2866
Copy link
Contributor Author

@liebman Thx for the info. Should be fixed. Delete .platformio/cache and .platformio/platforms to force the download of the changed version.

@Anthony-Lyric
Copy link

Anthony-Lyric commented Jun 6, 2022

@Jason2866
using:
platform = https://github.com/tasmota/platform-espressif32/releases/download/v.2.0.3/platform-espressif32-v.2.0.3.zip

Have you built a large project yet? I'm running into an issue where the rom size is not being read from the board file, and my project is running out of space, so I get eh following error:

Linking .pio\build\xxx\firmware.elf
Retrieving maximum program size .pio\build\xxx\firmware.elf
Checking size .pio\build\xxx\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
Error: The program size (1434797 bytes) is greater than maximum allowed (1048576 bytes)
RAM: [= ] 14.7% (used 78268 bytes from 532480 bytes)
*** [checkprogsize] Explicit exit, status 1
Flash: [==========] 136.8% (used 1434797 bytes from 1048576 bytes)

I can effect the ram high water mark in the board file, but ROM is stuck @ 1048576, nothing I do will change it. Also as an FYI the project builds fine in the IDF, so I know partition file is good, it's just plarformio's translation of the board file that I suspect.

Also as an FYI I'm using a ESP32-s3-wroom-1-m16n8, so should be able to set a 16Mb rom in board file.

@Jason2866
Copy link
Contributor Author

Easiest is to define your own boards.json which reflects your board to 100%
See documentation for. We do for Project Tasmota (175000 lines of code) as you can see here

@Anthony-Lyric
Copy link

Anthony-Lyric commented Jun 7, 2022 via email

@ivankravets ivankravets unpinned this issue Jul 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests