Allow libraries to be loaded from sketch / project folder

[vespakoen]

Hey!

This is an issue that must have wasted hours for many people, as some sources on the internet hint that there is a possibility to do it but it's actually not.

What is possible now:

• Add libraries to arduino's libraries folder (ARDUINO/lib/targets/libraries)
• NEW (as we can read here) Add libraries to ~/Documents/Arduino/libraries/ or My Documents\Arduino\libraries depending on your OS, actually after installing Arduino (using the built in install.sh on linux) it made an Arduino folder in my home directory, (~/Arduino) so, I should make a libraries folder in there?
• Add libraries to project directory, this seems to be the best "hack" but makes it very hard to find the actual application code.

What we all want

~/MyProject
~/MyProject/libraries/Fat16/Fat16.cpp
~/MyProject/libraries/Fat16/Fat16.h
...long list of other support files...
~/MyProject/MyProject.ino
~/MyProject/FileActuallyRelatedToTheProject.cpp

This doesn't seem to be possible without a (big ass) Makefile.

The problem with the current options:

• If I want to distribute my code, I have to add a install section that tells my users where to get and how to install libraries.
• If changes have to be made to libraries (to manually tree-shake as space is sometimes limited, anyone?) users have to determine the location of the libraries folder based on their arduino version and OS, download the modified libraries and put them into the correct folder.
• If 2 packages have a filename that collides, I have to move libraries out of the way and back when I need them again
• If I want to use 2 different versions of a library for different projects, what do I do?

Is there really no way we can have relative includes at the moment?

If not, please allow it or at the very least check if there is a libraries folder inside of the sketch / project directory and link those correctly

Pains

• http://arduino.stackexchange.com/questions/8651/loading-local-libraries
• http://stackoverflow.com/questions/14413372/creating-libraries-in-arduino-sketch-folder?s=1|0.4971
• http://stackoverflow.com/questions/4705790/keeping-all-libraries-in-the-arduino-sketch-directory?s=3|0.3257
• http://stackoverflow.com/questions/20413851/git-combining-multiple-repos
• http://stackoverflow.com/questions/7422841/using-arduino-ethernet-h-under-eclipse
• The list goes on, more here: http://arduino.stackexchange.com/search?q=libraries+sketch
• On the forum: https://www.google.com/search?q=arduino+libraries+subdirectory+site:forum.arduino.cc

[per1234]

It is now possible to do this. Note that the files outside the src subfolder are not compiled or used for library detection so you should place the libraries in the src subfolder of your sketch folder:

MyProject
|_MyProject.ino
|_src
  |_Fat16
    |_Fat16.h

and the include statement in MyProject.ino:

#include "src/Fat16/Fat16.h"

[angel333]

@per1234's solution doesn't seem to solve this issue because:

1. It probably shouldn't be used like that. From GCC docs (link):

   #include "file"
   This variant is used for header files of your own program...

2. While for some use cases #include "file" will work, it will fail for others. Consider using a library that uses another library (example). Such library will use the #include <file> syntax (rightfully so). Of course you could modify the #include statements in the library, but that seems to me like an ugly hack.

@vespakoen's points still apply, it is impossible to distribute libraries with the sketch using the official Arduino tools.

With arduino-cli it is now however possible to install libraries from CLI, making it easy to install libraries from a Makefile (example). It is possible to specify library's version, installing two versions of the same library is still not possible though.

[per1234]

It probably shouldn't be used like that. From GCC docs

Meh. I'm not buying it.

Of course you could modify the #include statements in the library, but that seems to me like an ugly hack.

A little inconvenient, but I think "ugly hack" is going too far.

vespakoen's points still apply

No. The src folder system provides exactly what vespakoen requested.

it is impossible to distribute libraries with the sketch with the official Arduino tools.

Come on. That's clearly a false statement. Perhaps it's possible to improve, but it absolutely is possible.

[angel333]

I've just tried it and, while the #include statements indeed have to be modified, it otherwise seems like a nice solution. I've dismissed it previously right away because it looked utterly clumsy and complicated with all the _ prefixes. Turns out I confused |_ for prefixes when there are in fact none (something like `- would probably have been a bit more understandable).

Should have asked. You're right and I'm sorry.

Btw, is this covered in any docs somewhere or one has to read the sources?

[per1234]

I've dismissed it previously right away because it looked utterly clumsy and complicated with all the _ prefixes. Turns out I confused |_ for prefixes when there are in fact none (something like `- would probably have been a bit more understandable).

Ah, I see now. It was my text representation of the folder structure. It indeed would be unacceptable if there was some required filename prefix. Sorry for the confusion.

Should have asked. You're right and I'm sorry.

No apologies necessary. I think it's possible there is a better solution to the general problem of bundling libraries with sketches. It's just that this particular feature request of vespakoen has been resolved. Other possibilities are:

https://github.com/arduino/Arduino/issues/7524

This is actually less applicable to this request since it won't allow different library versions to be used.

arduino/arduino-builder#223

I'm not sure I completely understand that PR, but this comment shows a lot of promise. If we could add a libraries subfolder to the sketch, and that folder was treated just as the other standard libraries folders ({sketchbook folder}/libraries, {IDE installation folder}/libraries, {hardware package folder}/libraries), then that would be an ideal solution because it would allow the libraries to be used without any change in #include directives, either in the sketch or the libraries. The sketch local libraries folder could have priority over the other folders so that there would be no possibility of filename collisions. I think that solution would perfectly address your requirements.

Btw, is this covered in any docs somewhere or one has to read the sources?

Unfortunately no. This is one of quite a few useful features that were never documented. I think the root problem is that there is not currently a good place to document it. I think the ideal location would be an "Arduino Sketch Specification", just as we already have for hardware packages, libraries, and Boards Manager JSON files. I brought that up in the developers mailing list some time ago but didn't get any response. I suppose I should probably just go ahead and do it and hope nobody minds.

[tbrianjones]

Is there really no standard way to include libraries in my sketch folder? This is driving me bonkers. I don't want my library relying on the setup of a specific computer. Being able to put a library folder in my sketch folder seems like an obvious solution.