Incompatibility issue with lwip napt on ESP32-S3 using Arduino framework

[Gspohu]

Board

ESP32-S3

Device Description

DevKitC

Hardware Configuration

Out of context

Version

v2.0.9

IDE Name

PlatformIO

Operating System

Kubuntu Linux 22.04

Flash frequency

1000

PSRAM enabled

yes

Upload speed

115200

Description

Hello @espressif team,

I have been developing a connected object project that is running successfully on an ESP32-S3 using the Arduino framework. I am now trying to add a functionality that involves lwip napt for a WiFi extender.

I tried prototyping this in Arduino but encountered an issue where I get the following error messages:

undefined reference to `ip_napt_enable_no'
undefined reference to `ip_napt_enable'

Interestingly, I had no such problems while implementing the same functionality with espidf.
For me, it is not possible to recode the whole firmware on espidf.

I have made several attempts to troubleshoot this issue to no avail:

• I tried including lwip as a "lib" or "component" in my project directory.
• I attempted to use Arduino as a component of espidf, but this was quite challenging and resulted in a performance drop in my Arduino code. Plus, the lwip version remained as the Arduino one.
• I tried replacing the lwip directory in the Arduino framework with the espidf one. As expected, this didn't work because Arduino uses precompiled libraries. I would love to understand the rationale behind this choice if anyone can help.
• I manipulated my platformio.ini to add the necessary compilation flags, but this was unsuccessful because, again, Arduino uses precompiled libraries.
• I also tried this: platform = https://github.com/tasmota/platform-espressif32/releases/download/v2.0.5.3/platform-espressif32-2.0.5.3.zip. However, it crashed my platformio, and I had to reset the software.

The issues I encountered seem to be similar to the ones described here, but I couldn't find a feasible solution:

• ip_napt_enable_no defined in lwip/lwip_napt.h, but not there in precompiled lib #6421
• Feature Request: lwip with NAT & IP Forwarding features #2826

I noticed there was supposed to be a fix in the Arduino lib builder (espressif/esp32-arduino-lib-builder#69), but it seems to be forgotten at the moment.

Interestingly, I found an implementation for ESP8266 (https://github.com/martin-ger/lwip_nat_arduino) but couldn't find an equivalent for ESP32.

For your reference, I am using platformio on vscode.

Is there any way to get lwip napt working with the Arduino framework on ESP32-S3? Any guidance or help would be greatly appreciated.

Thank you in advance for your support.

Sketch

#include "lwip/lwip_napt.h"

#include "lwip/app/dhcpserver.h"
// Initialize the NAT feature
ip_napt_init(IP_NAPT_MAX, IP_PORTMAP_MAX);

// Enable NAT on the AP interface
ip_napt_enable_no(1, 1);

// Set the DNS server for clients of the AP to the one we also use for the STA interface
dhcps_set_DNS(WiFi.dnsIP());

Debug Message

undefined reference to `ip_napt_enable_no'
undefined reference to `ip_napt_enable`

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[SuGlider]

It seems that espressif/esp32-arduino-lib-builder#69 should be reviewed and merged.
@me-no-dev - maybe add that to the 2.x and 3.0.0 branches?

[Gspohu]

@VojtechBartoska I just saw that you have assigned it to 3.0, which will be out in quite some time.

It's only one line in a configuration file, why wait so long?

[me-no-dev]

The problem is that NAPT also enables other things in ESP-IDF that cause performance issues for everyone, even when NAPT is off. We check from time to time to see if they have "fixed" that in IDF and when they do, we can add support for it.

[Jason2866]

@Gspohu I am the maintainer of the Arduino fork for Tasmota. We use NAPT and it does work. You can use latest Tasmota Core 2.0.9 with Platformio

platform = https://github.com/tasmota/platform-espressif32/releases/download/2023.05.00/platform-espressif32.zip

The version is trimed to use low flash and RAM. Do not expect logging messages from IDF.

[Gspohu]

Thanks for your message @Jason2866 , but what you suggest was my first try, but the result was not glorious.

Ok I was able to compile, but once the firmware was uploaded, my ESP32-S3 was rebooting in a loop, and other projects on my platformio were affected, for some reason, everything I compiled and upoloaded with my platformio, even when there was never any mention of your solution, gave an ESP32 that constantly crashed. The only solution I found was to delete the .plateformio folder, uninstall the extension, uninstall vscode and reinstall everything. Really strange.

[Jason2866]

@Gspohu You used a old build. At this time support for S3 in Platformio was faulty.
In main Platformio and our fork. Took my some amount of time to find. Provided PRs here and in Platformio repo to fix. S3 support is stable since a while.
You should pin the Platform version for your projects. If you don't do latest installed will be used with platform = espressif32 only! This info is recently added in platformio platform espressif32 github. To avoid such incompatible setups, i have decided to release always a new platform version when something "under the hood" changes. By doing this it is secured always using the compatible tools, toolchains and frameworks (Arduino core version).

[Gspohu]

@Jason2866 I just tried it and the problem is exactly the same. My ESP restarts in a loop

[Jason2866]

@Gspohu S3 needs 100% correct config. If not it bootloops.
What type of flash and PSRAM? QIO / OPI? It needs be correct for flash AND PSRAM.
And flash size too. You started with the zero error forgiving device ;-)
Just can say S3 and NAPT is working with our framework. Running Tasmota range extender on it.

[Gspohu]

@Jason2866 Okay, but I'm not sure if I understand.
My card is this one: https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html

When I use the classic tools, the configuration must be perfect, because I never had any problems like this.

I'm going to put an issue on Tasmota's repository, so as not to pollute the discussion here. We can continue to discuss this there

[Gspohu]

I decided to test the Tasmota web installer, thinking that maybe it was my platformio configuration that was the problem, or even my code (https://tasmota.github.io/install/), but the result is the same. There still seems to be some bugs with the ESP32-S3 support on Tasmota.

Finally, it's not possible to file an issue on the Tasmota repository. To finish on this topic here, could you point me to a place to discuss this topic @Jason2866 ?

[Jason2866]

@Gspohu It is a config thing. There is not just one S3 Devkit. The have variants with Flash and PSRAM types! You found us in Discord already. Let's talk there.

[RobertGnz]

The problem is that NAPT also enables other things in ESP-IDF that cause performance issues for everyone, even when NAPT is off. We check from time to time to see if they have "fixed" that in IDF and when they do, we can add support for it.

I recently compiled espressif idf 5.1.2 with napt feature using eclipse ide. The result was that napt works as expected provided you use esp_netif interface to enable napt. I did not noticed any overhead when enabling CONFIG_LWIP_L2_TO_L3_COPY which makes a copy of every tcp frame before passing a reference to it to L3 layer.
Furthermore I tested another buid disabling CONFIG_LWIP_L2_TO_L3_COPY but keeping enabled all other parameters needed for napt enabling. In this case no tcp frame was copied before passing a pointer to them to L3 layer. The result was that napt worked also perfectly as expected.
It appears that CONFIG_LWIP_L2_TO_L3_COPY is unnecessary to make napt working. So making a copy of tcp frames is unnecessary.
I believe that the CONFIG_LWIP_L2_TO_L3_COPY was initially programmed for debuging purpose during the development phase but now it is unnecessary so it can be removed.

[VojtechBartoska]

Hello, napt was implemented and this issue is supposed to be solved. I am closing it, if needed you can reopen.

[Rich968]

@VojtechBartoska I have updated esp32 board libraries and get: lwip/napt.h no such file or directory after including it in a project. Has the arduino esp32 lib builder been updated? (If that's relevant)

[Jason2866]

@Rich968 Yes, NAPT support has been added some time ago. The feature needs to be enabled in the sdkconfig. So yes there was an update in Arduino Lib Builder when NAPT support was added. You have to use actual Arduino Lib Builder and settings.
The feature is ONLY default for core 3.0.x. It is not enabled for core 2.0.x