build gcc with lto support

[nerdralph]

The included gcc wasn't built with the linker plugin, so enabling link-time optimization (-flto) fails.
dlopen:'.../arduino-1.6.1/hardware/tools/esp8266/xtensa-lx106-elf/bin/../libexec/gcc/xtensa-lx106-elf/4.8.2/liblto_plugin-0.dll': The specified module could not be found.
Lto not only reduces code size, it can significantly improve performance by doing constant folding for function calls like digitalWrite.
http://nerdralph.blogspot.ca/2014/04/gcc-link-time-optimization-can-fix-bad.html

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[rogerclarkmelbourne]

I think there is also an issue with the way gcc was compiled, so that the libiconv-2.dll, is required to be in the root of the Arduino IDE

This would prevent the repo being deployed by the new Boards Manager in IDE 1.6.2 or newer

Edit. libicon-v.dll problem still exists.

This should be fixed at the same time if possible.

[igrr]

Yes, thanks, looks like there are currently 3 issues that need to go into the toolchain:

• this one, adding lto support
• the dependency on libiconv
• need to rebuild ulibc without malloc and free

For Windows I used this build script:
https://gist.github.com/fpoussin/7ae55b5a5bd9a28ce21d

For Linux and Mac I used crosstool-NG (branch lx106-g++).

@nerdralph what options do we need to pass to build gcc with lto support?

[rogerclarkmelbourne]

Ivan,

Did you manage to fix the libiconv-2 dependency issue ?

PS. How did you run the script under windows Mingw?

[igrr]

No I didn't have a chance yet — I'm running OS X and for all the Windows stuff I have use my old laptop which I usually can access on weekends.

For the build, yes, I did use mingw.

[rogerclarkmelbourne]

OK
No worries
Thanks
Roger

[nerdralph]

It was a lot of work to figure out lto support for avr-gcc, so I imagine it
won't be any easier for the lx106. From what I recall, it depended not
only on enabling a configure option, but also on the "gold" linker plugin
version.
Right now I'm trying to figure out the lx106 instruction set including
which are 16/24 bit so I can write some code in hand tuned asm. After that
I'll look at the lto build if nobody figures it out in the meantime.
Ideally the expressif folks should release a new sdk with GCC 4.9.2 and
working lto.
On Apr 1, 2015 3:34 AM, "Ivan Grokhotkov" notifications@github.com wrote:

Yes, thanks, looks like there are currently 3 issues that need to go into
the toolchain:

• this one, adding lto support
• the dependency on libiconv
  https://github.com/rogerclarkmelbourne/arduino-esp8266/issues/2
• need to rebuild ulibc without malloc and free
  libc-provided malloc and free need to be replaced by os_malloc and os_free #11

For Windows I used this build script:
[https://gist.github.com/fpoussin/7ae55b5a5bd9a28ce21d]

For Linux and Mac I used crosstool-NG (branch lx106-g++).

@nerdralph https://github.com/nerdralph what options do we need to pass
to build gcc with lto support?

Reply to this email directly or view it on GitHub
#23 (comment).

[rogerclarkmelbourne]

Ivan

I've installed MinGW on my Windows machine, and I'm going to build (well I'm building) the using the script you linked to.

Do you have any idea's why we have the issue with libicobv-2.dll ?

I presume its a path issue?

The other version of the Xtensia compiler didn't have that issue, but I know the build switches were incorrect.

I presume the build.sh you linked to does build with the correct switches ?

I also presume it must be downloading a config file where somehow libiconv-2.dll path can be referenced ?

Anyway, I better leave it compiling, as I expect it may take several hours (or I can leave it on all night if necessary ;-)

Edit. I had some issues with Git not being installed by default with MinGW and have now fixed those, but I will need to try again tomorrow as its getting too late to continue tonight

If you have any tips for getting it to compile please let me know ;-)

[igrr]

There should be a way to tell gcc's configure script to link statically to libiconv. But I don't know which exact options are required for this though...

[igrr]

Perhaps this one might be useful: http://stackoverflow.com/a/14112368

[rogerclarkmelbourne]

Ivan

I've tried building on 2 different Windows 7 x 64 machines (more core i5 and one core2) and both just hang after a while in gcc :-(

This part of the script gives a warning ./ error

  find $XTDLP -maxdepth 1 -type d -name gmp-* | xargs -i mv -v {} $XTDLP/$GMP
  find $XTDLP -maxdepth 1 -type d -name mpfr-* | xargs -i mv -v {} $XTDLP/$MPFR
  find $XTDLP -maxdepth 1 -type d -name mpc-* | xargs -i mv -v {} $XTDLP/$MPC

but I think its OK, because its just a clumsy way to rename the file to what its already called (and fails on MinGW).

I guess I could try installing XP as a VM and see if it doesn't hang when I do that.
(I have an XP machine but its old and slow and doesnt have much ram so would take ages to compile I suspect)

Edit.
I just looked and Static libraries is supposed to be enabled already

--enable-static

and

--disable-shared --enable-static

are used

But perhaps not everywhere

[igrr]

Oh, right, it does hang quite a lot.
For me turning off parallel compilation helped:
JOBS=-j6 => JOBS=

[rogerclarkmelbourne]

Thanks

I changed to -j1 , and its no longer hanging

I'll see if it manages to complete a compile and then see what static link settings need to change, if any

I'll also need to do some research about Lto as recommended by @nerdralph

[rogerclarkmelbourne]

Ivan
Sorry to keep bothering you, but I'm now getting a build failure on line 164 because (the ../configure line)
because configure has not been created in

echo "Building first stage GCC"
cd $XTDLP/gcc-xtensa/build-1

dl\gcc-xtensa

did you make any other changes to the script that you can remember ?


if [ $RECONF -gt 0 -o ! -f .configured ]; then
  rm -f .configured
  ../configure --prefix=$XTTC --target=$TARGET --enable-multilib --enable-languages=c --with-newlib --disable-nls --disable-shared --disable-threads --with-gnu-as --with-gnu-ld --with-gmp=$XTBP/gmp --with-mpfr=$XTBP/mpfr --with-mpc=$XTBP/mpc  --disable-libssp --without-headers --disable-__cxa_atexit
  touch .configured
fi

[igrr]

Maybe I did... I'll check on my windows laptop tomorrow.

On Fri, Apr 3, 2015 at 10:46 AM, Roger Clark notifications@github.com
wrote:

Ivan
Sorry to keep bothering you, but I'm now getting a build failure on line
164 because (the ../configure line)
because configure has not been created in

echo "Building first stage GCC"
cd $XTDLP/gcc-xtensa/build-1

dl\gcc-xtensa

did you make any other changes to the script that you can remember ?

if [ $RECONF -gt 0 -o ! -f .configured ]; then
rm -f .configured
../configure --prefix=$XTTC --target=$TARGET --enable-multilib --enable-languages=c --with-newlib --disable-nls --disable-shared --disable-threads --with-gnu-as --with-gnu-ld --with-gmp=$XTBP/gmp --with-mpfr=$XTBP/mpfr --with-mpc=$XTBP/mpc --disable-libssp --without-headers --disable-__cxa_atexit
touch .configured
fi

Reply to this email directly or view it on GitHub
#23 (comment).

[rogerclarkmelbourne]

Ivan,

Its OK. I worked out what was wrong.
Somehow the gcc-xtensa is empty apart from the build1 and build2 folders created by the script

I'm not sure how this happened.

Anyway, it looks like its a big repo, so I will leave it downloading and look again tomorrow

Apart from that, the build now seems OK,

If you do run your windows laptop, could you tell me if you get compiler warnings all the time for "llabs" not being defined ?

Thanks
Roger

[rogerclarkmelbourne]

I'm not sure why this works, but Sampreet's repo works on IDE 1.6.3 without needing libiconv-2.dll in the IDE root directory.

Anyway. On the build front. g++ and a few other things are still not getting built for me, so I'll need to track down where in build.sh creates them.

Re: LinkTimeOptimization

It looks like currently adding -flto makes no difference to the output size, but it still compiles OK.

But as @nerdralph has indicated, gcc would need to be built with LTO enabled as part of the build, and at the moment, its not too clear how to do this, or whether its possible on the Xtensa version of gcc

I found this reference https://gcc.gnu.org/wiki/LinkTimeOptimization

Which seems to say that the LTO version is a branch from the main GCC repo

But I can't see a branch for LTO on https://github.com/jcmvbkbc/ so I guess the only thing would be to try adding the --enable-lto and see if it gets passed down into the actual compile in the same way as --enable-static is supposed to.

However at the moment I think this is not a top priority, as its probably a lot of work / waiting for compiles to finish etc, for possibly not too much difference for most people

[nerdralph]

Gcc 5.0 has a lot of new IPA optimizations, even without lto:
https://gcc.gnu.org/gcc-5/changes.html
Hopefully Max moves to 5.0 from 4.8.2 soon.
If you haven't read Honda's blog, it explains his work on lto.
http://hubicka.blogspot.ca/2014/04/linktime-optimization-in-gcc-2-firefox.html
On Apr 4, 2015 2:47 AM, "Roger Clark" notifications@github.com wrote:

I'm not sure why this works, but Sampreet's repo works on IDE 1.6.3
without needing libiconv-2.dll in the IDE root directory.

Anyway. On the build front. g++ and a few other things are still not
getting built for me, so I'll need to track down where in build.sh creates
them.

Re: LinkTimeOptimization

It looks like currently adding -flto makes no difference to the output
size, but it still compiles OK.

But as @nerdralph https://github.com/nerdralph has indicated, gcc would
need to be built with LTO enabled as part of the build, and at the moment,
its not too clear how to do this, or whether its possible on the Xtensa
version of gcc

I found this reference https://gcc.gnu.org/wiki/LinkTimeOptimization

Which seems to say that the LTO version is a branch from the main GCC repo

But I can't see a branch for LTO on https://github.com/jcmvbkbc/ so I
guess the only thing would be to try adding the --enable-lto and see if it
gets passed down into the actual compile in the same way as --enable-static
is supposed to.

However at the moment I think this is not a top priority, as its probably
a lot of work / waiting for compiles to finish etc, for possibly not too
much difference for most people

Reply to this email directly or view it on GitHub
#23 (comment).

[igrr]

@rogerclarkmelbourne were you able to make some progress with the gcc build?

[rogerclarkmelbourne]

Hi Ivan

I tried on a Window XP VM, but it kept hanging like it did before I change the jobs settings

But the jobs settings was -j1
I also tried
jobs=

But nothing worked :-(

I have been too busy with the Boards Manager issues on my other repo STM32

If you send me your XP build script I can try it, but apart from that I don't know how to proceed.

PS. I'm not sure if we also need to try gcc 5, but I think it will probably have its own problems ;-)

[igrr]

I checked my build script and it's no different from the one linked above, aside from -j1 option.
Guess i need to get myself a windows machine, perhaps some cheap option from Azure Cloud will do.

[rogerclarkmelbourne]

Ivan,

I have been using Azure for work, but its expensive and not very fast unless you pay $$$$ .

I'm not sure if its possible to request a 32 bit machine on Azure.

I think you would be better off dual booting your computer or using a VM.

Do you run Linux or OSX?

I ran an vm on OSX with XP int it. ( I think I used virtual box )

But I also tried XP 32 bit under VM on windows 7 x64, and the compiler keeps hanging :-(

I guess I should find a copy of Windows 7 , 32 bit and run that in the VM and see if the compiler still hangs.

But perhaps the compiler doesn't work in the VM ( virtual box), and perhaps I need to try VirtualPC

[rogerclarkmelbourne]

@nerdralph

What version of MinGW are you using ?

I have been using http://sourceforge.net/projects/mingw/postdownload?source=dlp but I noticed there is another version called MinGW-W64 which seems to be gaining traction and claims to be contain the latest / later releases of gcc

However, it doesn't seem to have bash shell so I can't fun the script that Ivan uses

Anyway, I've made a new Windows 7 X86 VM using Microsoft Virtual PC and I will see if that runs Ivans' script any better than the Oracle Virtual Box VM running XP 32 bit.

[nerdralph]

Roger,
I'm using the same version. Yesterday I built avr-gcc-4.9.2 on mingw
(since I'm experimenting I figured I'd build something I can use). Builds
can hang if you use parallel make (-j>1) or don't use -D__MINGW_ACCESS.
I also built the 20150405 snapshot on Linux. My build script downloads
gmp, mpc, and mpfr sources and links them in the gcc source dir. I still
haven't figured out the xtensa target configuration options from the gcc
docs, and may have to dig into the source.
On Apr 12, 2015 2:25 AM, "Roger Clark" notifications@github.com wrote:

@nerdralph https://github.com/nerdralph

What version of MinGW are you using ?

I have been using
http://sourceforge.net/projects/mingw/postdownload?source=dlp but I
noticed there is another version called MinGW-W64 which seems to be gaining
traction and claims to be contain the latest / later releases of gcc

However, it doesn't seem to have bash shell so I can't fun the script that
Ivan uses

Anyway, I've made a new Windows 7 X86 VM using Microsoft Virtual PC and I
will see if that runs Ivans' script any better than the Oracle Virtual Box
VM running XP 32 bit.

Reply to this email directly or view it on GitHub
#23 (comment).

[nerdralph]

That should be __USE_MINGW_ACCESS
On Apr 12, 2015 9:33 AM, "Ralph Doncaster" ralphdoncaster@gmail.com wrote:

Roger,
I'm using the same version. Yesterday I built avr-gcc-4.9.2 on mingw
(since I'm experimenting I figured I'd build something I can use). Builds
can hang if you use parallel make (-j>1) or don't use -D__MINGW_ACCESS.
I also built the 20150405 snapshot on Linux. My build script downloads
gmp, mpc, and mpfr sources and links them in the gcc source dir. I still
haven't figured out the xtensa target configuration options from the gcc
docs, and may have to dig into the source.
On Apr 12, 2015 2:25 AM, "Roger Clark" notifications@github.com wrote:

@nerdralph https://github.com/nerdralph

What version of MinGW are you using ?

I have been using
http://sourceforge.net/projects/mingw/postdownload?source=dlp but I
noticed there is another version called MinGW-W64 which seems to be gaining
traction and claims to be contain the latest / later releases of gcc

However, it doesn't seem to have bash shell so I can't fun the script
that Ivan uses

Anyway, I've made a new Windows 7 X86 VM using Microsoft Virtual PC and I
will see if that runs Ivans' script any better than the Oracle Virtual Box
VM running XP 32 bit.

Reply to this email directly or view it on GitHub
#23 (comment).

[nerdralph]

Just tried another avr-gcc build on MinGW, with __USE_MINGW_ACCESS, and make -j2, and it locked up again. This is with the stock mingw gcc-4.8.1 and binutils. Looks like parallel builds are out for now.
Looking through gcc's config files, I see config/mh_mingw already defines __USE_MINGW_ACCESS.

[rogerclarkmelbourne]

Ralph

Thanks for the info.

I tried using Microsoft Virtual PC and a copy of Windows 7 x86, instead if virtual box, but the shared drives with the host are not working, ( aaarrrghhh), which makes it a pain to use, so I may go back to Oracle Virtualbox but with Window 7 x86 rather than XP.

Where in the script do you add __USE_MINGW_ACCESS ?

[nerdralph]

Roger, first look for a configure option for gcc... Perhaps
--disable-shared.
https://gcc.gnu.org/install/configure.html
Also look at Max's crosstool-NG xtensa repo, which seems to encapsulate the
build of binutils, gcc, and possibly libc.
On Apr 13, 2015 6:23 PM, "Roger Clark" notifications@github.com wrote:

Ivan and Ralph

It looks like I need to get lib iconv from here.
https://www.gnu.org/software/libiconv/#downloading

And build it as a static library ( which on windows doesn't seem to be the
default build target)

Then the Makefile for gcc needs to be changed to link the static library.

Which will mean somehow changing the gcc Makefile, but the Makefile is
actually created by Automake.

So I presume the correct way to do this is to somehow change the Automake
template, but this is all new to me.
I have a basic understanding of Make files, but Automake is new to me

Reply to this email directly or view it on GitHub
#23 (comment).

[rogerclarkmelbourne]

Ralph,
I'll need to double check, but I was pretty sure that --disable-shared was already in the build scripts

It looks like esp-newlib contans iconv sources

BTW.
The other possibility is to use PFalcon's build process as outlined here

https://github.com/pfalcon/esp-open-sdk

However its not clear how to build this on Windows.
PFalcon mentions "Windows with Cygwin"

I guess perhaps I should post an issue on that repo to see if there are windows build instructions

[igrr]

@rogerclarkmelbourne I got my windows laptop back and looks like I'm making progress.
Downloaded and build libiconv with --disable-shared flag, added an --with-libiconv-prefix=/path/to/my/libiconv option to gcc configure call and also removed libiconv-dev package from MinGW (the last one was the key - with this package installed, configure would pick the shared libiconv from MinGW). Checked the resulting xtensa-lx106-elf-gcc with Dependency Walker, and looks like it doesn't depend on libiconv-2.dll any more!

[rogerclarkmelbourne]

Ivan

OK.

I'm not sure if you saw the issue has been fixed in Sandeep's repo, as someone found if you add the dll in 2 different places, it solves the problem

But if you have totally removed the dependencies that's an even better solution

[igrr]

No I haven't seen that solution, will check.
On May 5, 2015 08:58, "Roger Clark" notifications@github.com wrote:

Ivan

OK.

I'm not sure if you saw the issue has been fixed in Sanpreet's repo, as
someone found if you add the dll in 2 different places, it solves the
problem

But if you have totally removed the dependencies that's an even better
solution

Reply to this email directly or view it on GitHub
#23 (comment).

[rogerclarkmelbourne]

https://github.com/sandeepmistry/esp8266-Arduino/commit/0ce775c41ec79e64895a4e288441bb539f60bf4e

PS. I presumed that Sandeep had contacted you ;-(

[igrr]

That way is actually preferrable because the code is not statically linked into multiple gcc executables (gcc, gcc-version, g++, c++, etc). Will update my toolchain archive accordingly.

[rogerclarkmelbourne]

Cool... Glad to be vaguely useful ;-) (as I failed to recompile the compiler )

[nerdralph]

Based on builds I've done of gcc on Linux, static linking had less of an
impact on size than I expected. Only the parts of the shared library that
are used by the executable get linked in, whereas with shared libs, you
have the whole library including the unused parts. Also many of the gcc
executables are links to the same file (like c++ and g++).

On Tue, May 5, 2015 at 3:10 AM, Ivan Grokhotkov notifications@github.com
wrote:

That way is actually preferrable because the code is not statically linked
into multiple gcc executables (gcc, gcc-version, g++, c++, etc). Will
update my toolchain archive accordingly.

Reply to this email directly or view it on GitHub
#23 (comment).

[AutomationD]

I'm working on building an alternative sdk: kireevco/esp-alt-sdk, which is based on similar concept as esp-open-sdk, but is meant to work on all plarforms, including Windows. It's not using crosstool-ng, and should work on mingw32,64 and probably cygwin. Currently I have a build working, some people reported errors, some people reported it working. I'm experimenting with https://github.com/fpoussin/esp-binutils vs https://github.com/jcmvbkbc/binutils-gdb-xtensa (latter has gdb in it).

Any thoughts/testing/contributions are welcome!

[nerdralph]

I think the ideal would be taking Max's work and creating a script to add
the lx106 files to a stock gcc source tree. 4.8 is no longer maintained.
4.9 is the minimum version you should be at, and preferably 5.2.
On Dec 15, 2015 9:29 PM, "Dmitry Kireev" notifications@github.com wrote:

I'm working on building an alternative sdk: kireevco/esp-alt-sdk, which is
based on similar concept as esp-open-sdk, but is meant to work on all
plarforms, including Windows. It's not using crosstool-ng, and should work
on mingw32,64 and probably cygwin. Currently I have a build working, some
people reported errors, some people reported it working. I'm experimenting
with https://github.com/fpoussin/esp-binutils vs
https://github.com/jcmvbkbc/binutils-gdb-xtensa (latter has gdb in it).

Any thoughts/testing/contributions are welcome!

—
Reply to this email directly or view it on GitHub
#23 (comment).

[AutomationD]

@nerdralph not sure what do you mean, can you clarify please?

[nerdralph]

Max's gcc-xtensa is based on gcc 4.8.2.
The lx106 is not a supported target in the stock gcc sources.
Take Max's mods to the 4.8.2 tree and create a script or patch file so you
can add lx106 support to the stock 4.9.3 and 5.2 sources.
On Dec 15, 2015 10:09 PM, "Dmitry Kireev" notifications@github.com wrote:

@nerdralph https://github.com/nerdralph not sure what do you mean, can
you clarify please?

—
Reply to this email directly or view it on GitHub
#23 (comment).

[AutomationD]

So, you want to take https://gcc.gnu.org/releases.html, and add lx106? That's pretty tough step, I think it's been maintained here https://github.com/jcmvbkbc/gcc-xtensa/tree/lx106-5.1?

[AutomationD]

By the way, how crosstool-ng is structured? I can't find a clear set of instructions how they build binutils?

[nerdralph]

You were asking for feedback; you didn't say it had to be easy. :-)
Optimization in 4.8 without lto is pretty lame.
If ipa-ra in 5.2 can work with the lx106, I expect you'd see 10-25%
improvement in speed and code size.
On Dec 15, 2015 10:16 PM, "Dmitry Kireev" notifications@github.com wrote:

So, you want to take https://gcc.gnu.org/releases.html, and add lx106?
That's pretty tough step, I think it's been maintained here
https://github.com/jcmvbkbc/gcc-xtensa/tree/lx106-5.1?

—
Reply to this email directly or view it on GitHub
#23 (comment).

[AutomationD]

@nerdralph

You were asking for feedback; you didn't say it had to be easy.

:-) fair enough:)

If ipa-ra in 5.2 can work with the lx106, I expect you'd see 10-25%
improvement in speed and code size.

Well, that sounds great, really!

[nerdralph]

I didn't get very far into crosstool-ng before I decided it was an option I
didn't want to pursue.
On Dec 15, 2015 10:24 PM, "Dmitry Kireev" notifications@github.com wrote:

By the way, how crosstool-ng is structured? I can't find a clear set of
instructions how they build binutils?

—
Reply to this email directly or view it on GitHub
#23 (comment).

[AutomationD]

@nerdralph that's the issue I'm trying to solve with esp-alt-sdk. Crosstoon-ng is way far from working ok on Windows, and I need to re-do the logic that is under ./ct-ng xtensa-lx105...

[AutomationD]

@nerdralph just for fun, I'm trying to build binutils, gcc for xtensa target... we'll see how it goes.

[milkpirate]

Dont know if this issue is still valid but since I was searching for a newer GCC version (better constexpr handling) for the ESP8266 I stumbled across it and a little later across that:

Binutils	GCC	libc	GDB	Download link
2.24	5.2.0	Newlib 2.0.0	7.11	esp8266-gcc5.2.0-r10.exe

Link: http://gnutoolchains.com/esp8266/

That might be the desired GCC 5.2.

[devyte]

@igrr we discussed briefly moving to a newer toolchain, namely 5.2, which is the same as mentioned here. Would that address the issues discussed above?
If so, I suggest tracking here, but I'd change the issue title.

[igrr]

While this issue does depend on updating to 5.2.0, there's (much) more to LTO support than just a toolchain update. Some core/libraries changes will be required for binaries built with LTO to actually work.