Ability to specify additional compiler flags [imported]

[cmaglie]

This is Issue 421 moved from a Google Code project.
Added by 2010-11-30T03:12:51.000Z by steamra...@yahoo.com.
Please review that bug for more context and additional comments, but update this bug.

Original labels: Type-Enhancement, Priority-Medium, Component-IDE

Original description

A simple option in preferences.txt to change compiler flags instead of having them hard-wired in would be very nice.

A way to change it in the GUI would be even better.

[matthijskooijman]

Here's a summary for the original Google Code issue. It talks about:

• Wether to replace the default options or to add to them. Since most options on the gcc commandline can be undone by later options, adding them seems sufficient
• Using cflags, cppflags, asmflags and perhaps ldflags.
• Applying these flags to the core, libraries and sketch.
• Storing these options per-board in boards.txt or as a user-preference in preferences.txt.

There are a few links to possible implementations in the original issue as well (but nobody really commented on those).

Finally, one of the last comments seems quite insightful to me, so I'll repeat that one here verbatim:

The easy solution is to read the default compiler options from the
preferences file. Hardcoding important settings like these is always
bad practice. That way if someone find they really have to change it
they can keep going with a text editor.

Splendid would be to have a text entry box in the GUI, separately for
C/C++/asm (Paul in #2 knows what he's talking about), for options that
are added to the default options. The ldflags one is not crazy, it
allows more flexibility at the linker stage, e.g. to use other libraries
(those libXXX.a, not what Arduino calls library).

The real problem seems to be that the need for changing compiler options
is not understood. The Arduino IDE must get the prize for the only tool
not being able to do so. It's not professional - unless of course the
design goal is that the IDE is only used by beginners or for basic jobs.

IMO the IDE's job is to provide a default, but not to be in the way of
users who need to change options. I am ditching the IDE now because it
doesn't get the job done. The job requires different compiler options.

Don't try to hit all the possible compiler options people might want
with a few radio buttons, unless you like lost causes. Look at some
other tools at the profesional level, e.g. eclipse. It has selection
boxes for some common categories, like code optimisation level, macros,
basic warnings - and it has a catch-all box for the rest. I'm certain
there would be mutiny if that box wasn't there.

Some of my requirements (remember this is not and never will be
complete):

• Better warnings. This finds problems early. I read that Arduino is
  often used as a teaching tool. For a teaching example the state of the
  core software is quite shocking. Good examples provide no compiler
  warnings, or at least their header files don't bombard the user with
  warnings if the user decides to set the warning level to maximum for
  their own programs. I strongly suggest Arduino developers run their
  wiring code through maximum warning levels - it shows just how bad it
  is. It might even find a bug.
• Different optimisations. I claim the final decision of where and how I
  want my source to be optimised for myself, because I know my project
  best. I wanted to test LTO in newer compilers, but it needs more
  options. The IDE, while allowing to use any compiler (by setting up
  $PATH appropriately), is useless for this.
• Macro definitions with -D. I want to be able to make generic code that
  can be re-used in multiple projects, but there isn't code space for
  some code branches that are never used, and gcc isn't good enough to
  eliminate them automatically. It needs pre-processor help. Obviously
  this makes it unsuitable for libraries, that's a compromise I have to
  accept.

You could of course argue that those who need other compiler options use
something other than the IDE anyway. On the other hand implementing this
feature isn't exactly difficult. I would have found it very useful over
a year ago.

[matthijskooijman]

Now, I also have some opinions of my own:

• It seems to me that storing compiler options inside preferences.txt file is nice, but not sufficient. Some options might be associated with a particular sketch and should be stored inside the sketch (directory) (mostly defines that influence library parameters like buffer sizes, but also options like -std=c++11 which might be needed for some sketches.
• Most of the stuff can probably be done by adding to the compiler options, but I think it might be useful to have at least some way to replace the default options. I think this would only make sense inside the preferences.txt file, since that's where a user might want ultimate control over what is happening. If a specific board needs to replace the defaults, then the defaults probably need changing, for example. This could probably be implemented by making a "base_cflags" option that replaces the hardcoded default cflags and then having multiple "append_clags" options (in boards.txt, preferences.txt and the sketch) which all get appended to the base options.
• In addition to this, it would be even cooler (and more novice-proof) if libraries could define a set of compiletime options and these could be set through the IDE (similar to the Linux kernel's Kconfig files and menuconfig interface). However, since this only caters for the library options usecase, the general compiler options infrastructure should be also implemented.

[noisymime]

Anyone had a chance to look over this recently? There were patches proposed back in the Google Code bug that looked promising.

Whilst I agree with matthijskooijman (Particularly around sketch specific options), just having some form of user accessible option for compile time options would be a big step up from what we have now.

[EternityForest]

I think that -D definitions are probably one of the most common use cases, and should be handled separately from other compiler options. Ideally any sketch or library would be able to have an options.txt file, which would define a list of options that could be changed in a simple dialog box. Either that or a special pragmatic that would let you add things to the dialog from code directly.

Since that would be difficult, the easy way might just be to allow the user to create a file in a tab just like a code file containing compiler directives. It could be as simple as concatenating all the lines of the file, putting spaces in between and escaping anything within a line that needs escaping, and tacking it onto the command line, or it could be a more structured file, something like YAML, that would let you override things in boards.tx, like if you had a sketch that is meant to run with a different clock rate.

But overriding boards.txt might just be useless bloat because of the overlap with sketchbook/hardware/boards.tx

This could be a big deal for libraries. Libraries could become a lot more customizable and faster.

The point by matthijskooijman about options going with sketches is important. Nobody likes to modify a library just for one sketch. If the options file was just in another tab, then library writers are free to write in hooks to change stuff.

[GDV0]

HI EternityForest,

Some months ago, I worked on a patch to allow sketch to support library options.
You can retreive it in Pull request #1808.

[EternityForest]

The patch looks good to me. Personally I'd prefer to add the configuration file to my sketch manually, and just have a single optional configuration file per sketch that affected all libraries, but your way is much easier for beginners I think.

[ElectricRCAircraftGuy]

Hi does anyone know what the compiler settings are currently set to in the Arduino IDE and where can I find this?

[ElectricRCAircraftGuy]

I think I found it, though I don't know what I'm looking at exactly.
"C:\Program Files (x86)\Arduino\hardware\arduino\avr\platform.txt"

[matthijskooijman]

Yup, that's the right place. See https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5---3rd-party-Hardware-specification for (slightly out-of-date) documentation.

[anders-liu]

It's really a long-term issue....

But I still want to vote it, because the application-wise definition (-D option) is really important for me.

I'm going to write a library, and I want user to use #define MY_FLAG to control which will be compiled in my library. I don't want use run time parameter which increase program size and many code never runs in the application lifetime.

If this's hard to be done in IDE, could you add some special 'higher level' comments in code? For example, let user can define compiler options in code (*.ino file) like this:

/// [!compiler-options] -D MY_FLAG

#include "library.h"

void setup(){}

void loop() {}

You can parse it before send code to compiler, and append/replace the option set in code explicitly.

[facchinm]

@anders-liu could something like this work? facchinm/arduino-builder@904250f

[matthijskooijman]

@facchinm, this looks overpowered to me: AFAICS it allows setting any build property, which is really flexible, but also allows overriding e.g. the entire compiler commandline, which allows executing arbitrary commands as well (e.g. rm -rf /, or a root shell, or a backdoor installer, etc.). That's not quite what I expect when I compile someone else's code.

I'm not quite sure where the balance lies: Limiting to just #defines might be good (and perhaps make the syntax more specific for that (#arduino-define FOO=bar) rather than accepting the full properties and trying to filter that. Perhaps more things make sense than just define. Things like the board to compile for would make sense, though that should probably be handled by the IDE (e.g. preload the board when opening the sketch, but still allow changing it afterwards instead of silently overriding the select board on all builds).

[facchinm]

Totally agree, right now that approach is very insecure but I like the cgo approach (see https://golang.org/cmd/cgo/#hdr-Using_cgo_with_the_go_command).
The directives will change from

// #cgo CFLAGS: -DPNG_DEBUG=1
// #cgo amd64 386 CFLAGS: -DX86=1
// #cgo LDFLAGS: -lpng

into something like

// #arduino define: PNG_DEBUG=1
// #arduino arduino:avr define: X86=0

which is super readable and debuggable.
Need to discuss about that, anyway 😛

[anders-liu]

@facchinm First your efforts are great. But why do I need to involve such heavy build process? I just making some toys... If I need the heavy works, why don't I just switch to an IDE, such as Atmel Studio.

[Himura2la]

It's very strange that I can not configure libraries even through the build flags. Vote for this issue

[hank]

I need to specify the following easily (currently I'm modifying platform.txt directly, which is terrible because it's not portable):

compiler.cpp.flags=-c -Os {compiler.warning_flags} -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -funsigned-char -funsigned-bitfields -fpack-struct -fshort-enums -finline-limit=3 -fno-inline-small-functions -mcall-prologues -Wl,--relax -fno-tree-scev-cprop

I like the idea above that would allow me to do something like the following in the sketch:

// #arduino compiler.cpp.flags: -c -Os {compiler.warning_flags} -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -MMD -flto -funsigned-char -funsigned-bitfields -fpack-struct -fshort-enums -finline-limit=3 -fno-inline-small-functions -mcall-prologues -Wl,--relax -fno-tree-scev-cprop

Making them direct overrides of platform.txt keys would be my preference. That way, it's very obvious what you're modifying. You could also support a simple append functionality if someone just wanted to add a flag (which is probably what I actually need):

// #arduino append compiler.cpp.flags: -funsigned-char -funsigned-bitfields -fpack-struct -fshort-enums -finline-limit=3 -fno-inline-small-functions -mcall-prologues -Wl,--relax -fno-tree-scev-cprop

Syntax is up for debate; as long as I can do this in the .ino or a .cpp file, I'm pretty comfortable with anything.

[ArizonaClark]

FWIW, I have a somewhat different application for the ability to be able to specify "-D<name>=<value>" than the uses others have mentioned.

I have multiple very similar hardware units to program that require only minor differences in the source code (which is currently over 1000 lines long), e.g., each has to have a unique IP address, and some have different sensors connected. I'd very much like to have something like
#if UNIT == UNIT01
#define MY_IP "192.168.0.101"
#elif UNIT == UNIT02
#define MY_IP "192.168.0.102"
and so on. (There are other relatively minor differences besides this simple example.)

Based on a half century of programming, I'd suggest an "Additional Compiler Options" under "Tools" in the IDE.

[d-a-v]

This is a very interesting idea @facchinm

Need to discuss about that, anyway

// #arduino define: PNG_DEBUG=1
// #arduino arduino:avr define: X86=0

which is super readable and debuggable.

Would Arduino maintainers agree with such a PRoposal or has anybody already started to be working on such a great feature ?

[d-a-v]

I have a proposal for this issue.
Please check arduino/arduino-cli/pull/1117 arduino/arduino-cli#1517 arduino/arduino-cli#1524 and arduino/tooling-rfcs#14