esp8266 Arduino package can't be used as referenced package

[JAndrassy]

Arduino IDE 3rd party Hardware specification section "Referencing another core, variant or tool" specifies that a boards package can be referred from custom hardware definitions or other boards packages.

esp8266 package can't be used now as referred, because platform.txt file entries use variable {runtime.platform.path}, which points to the folder of selected board's boards.txt file. In case of a referring definition it is not the right folder to access sdk and tools.

solved in PR #5770

[JAndrassy]

the PR did not solve it. the solution was to use {build.core.path}/../.. instead of {runtime.platform.path} but in git version not all platform.txt keys can work with {build.core.path}.

other option is to do the replace as part of release preparation

[JAndrassy]

@d-a-v David, after the new release I again try to solve the "referred package" issue. The main problem are paths to the tools folder in platform folder (hardware/esp8266/esp8266/tools) used in platform.txt as {runtime.platform.path}/tools/
I discovered that Arduino IDE creators thought about need of a folder of this kind, but they prescribed a name system to it and then it is accessible as {build.system.path}. This works good if platform is used as referred package.

So if the tools folder would be renamed to system and all (note*) {runtime.platform.path}/tools would be replaced to {build.system.path} we would get closer to make Arduino esp8266 core work as referred.
The {build.system.path} works good in git install too.

note*: there is a new problem in 2.5.1. the

tools.esptool.upload.network_pattern="{network_cmd}" "{runtime.platform.path}/tools/espota.py"

doesn't evaluate anything else then the problematic runtime.platform.path

arduino-sam package (Due) uses the system folder https://github.com/arduino/ArduinoCore-sam/tree/master/system

[devyte]

@JAndrassy I suspect this isn't an issue with wide impact, so you're the main interested party. You started a proposal with #5770, but then dropped it, and it's not clear to me why. Will you be continuing your work, or should this issue just be closed?

[JAndrassy]

the maintainers should be interested, because now every manufacturer of a esp8266 board must add the board to this package and can't maintain own board definitions.

it is hard to work on this if the platform.txt file changes so much like in last months.

would the maintainers accept renaming of the tools folder to system? it would solve problems with the path to tools because system is what Arduino builder expects as name for this kind of folder.

[devyte]

@JAndrassy please make your proposal in a PR, or update #5770. We can discuss further there.

[JAndrassy]

I would need a release version of the current platform.txt. How is it transformed for release?

[d-a-v]

Only version will change in that file, in the next release.

[JAndrassy]

no. there is some edit not tracked in git. tools removed, paths of tools changed

and this line was removed in packaged platform.txt in 2.5.2

recipe.hooks.core.prebuild.2.pattern="{runtime.tools.python.path}/python" "{runtime.tools.makecorever}" --build_path "{build.path}" --platform_path "{runtime.platform.path}" --version "unix-{version}"

[d-a-v]

You are right, sorry.
Changes are made by package/build_boards_manager_package.sh.
You can run it locally, it will create a subdir with the installable archive including the updated platform.txt in it.

[JAndrassy]

So when I opened this issue, esptool was installed as IDE tool, next to mkspiffs and xtensa-lx106-elf-gcc. But then the upload tool changed and is with SDK in tools folder next to core folder. And this path can't be used from a referring package to run commands, because there is no build variable which would evaluate to this path for a command. That is why my PR to solve the referring of the package failed.

So first thing to solve would be to make the upload.py & co installable as IDE tool like mkspiffs or the old esptool.

[JAndrassy]

Of course the problem with the build variables evaluation at Upload is a problem of the Arduino IDE. I investigated it and reported it in May but then new version of IDE doesn't have it fixed.
https://github.com/arduino/Arduino/issues/8886

@earlephilhower please could the py scripts from tools folder be 'external' tools in boards manager package (like the old esptool was or mkspiffs is)?

[d-a-v]

@JAndrassy is that what you need achieved by #5970 #5770 ?
Why did you close it ?
Can you explain what is a referred package and when it is needed ?