Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support bundling libraries with a sketch #1255

Open
3 tasks done
hoblins opened this issue Jul 4, 2018 · 18 comments · May be fixed by #2514
Open
3 tasks done

Support bundling libraries with a sketch #1255

hoblins opened this issue Jul 4, 2018 · 18 comments · May be fixed by #2514
Labels
topic: code Related to content of the project itself type: enhancement Proposed improvement

Comments

@hoblins
Copy link

hoblins commented Jul 4, 2018

Describe the request

🙂 It will be easier for sketch developers to share a sketch as a package that includes library dependencies, avoiding the need for each user to install the library dependencies.
🙂 Library dependencies of the sketch can be controlled, allowing them to be pinned to the specific versions the sketch was intended to be used with, independent from the version of any globally installations of the library. As an example there are major changes to the API of the very popular "ArduinoJson" library in version 6 and sketches only work with one or the other major version series of that dependency.

Describe the current behavior

The common approach to bundling libraries with a sketch is to dump the library code files into the sketch root folder.

Arduino CLI version

a58d5ad

Operating system

All

Operating system version

All

Additional context

  1. As a workaround in the preferences the sketchbook location (directories.user) can be set to the sketch folder. But this has to be done manually and changed every time another program is handled. So there could be some option for this to be set in the .ino file.
  2. In the build folder a file build.options.json is generated automatically.
    There are some options where additional library folders could possibly be added?
    E.g.:
    • builtInLibrariesFolders
    • otherLibrariesFolders

Additional Requests

Issue checklist

  • I searched for previous requests in the issue tracker
  • I verified the feature was still missing when using the latest nightly build
  • My request contains all necessary details
@matthijskooijman
Copy link
Collaborator

I believe there are open issues about this already, did you look for them?

In any case, I'd really like to see the IDE support a libraries folder (as well as a hardware folder) inside the sketch directory, as you suggest here. That really helps to distribute a self-contained sketch, which is useful in some cases.

@per1234
Copy link
Contributor

per1234 commented Mar 26, 2019

I looked for other open issues and found a couple related things, though none that would make this a duplicate:


The addition of recursive compilation of the src subfolder of the sketch folder does give us this feature to some extent. However, I suspect @hoblins is already aware of that option from their browsing of the forum discussions and found it to be unsatisfying.

The problem with bundling libraries in the src subfolder is that all use of the #include <foo.h> syntax in the bundled libraries for files in the src subfolder, which would work fine if the libraries were installed normally, must be changed to use relative paths to those files. Best practices would be for libraries to not use the #include <foo.h> syntax for internal dependencies anyway so I don't much mind changing those, but when you have dependencies between bundled libraries it gets ugly (e.g. #include <foo.h> -> #include "../../foo/src/foo.h").

@waza-ari

This comment was marked as duplicate.

@probonopd

This comment was marked as off-topic.

@alandsidel
Copy link

I want to add a +1 here too. I'm working on a series of sketches all utilizing the same set of core private headers, and having to make duplicate copies of them in every project because the compiler doesn't understand relative paths is a real pain in the neck. I have maybe half a dozen projects that need to #include "../../includes/somefile.h" and there doesn't seem to be any way to convince the arduino compiler to do this. Even just reverting to #include "somefile.h" and then making that file a symlink to the real file doesn't work.

@matthijskooijman
Copy link
Collaborator

I want to add a +1 here too. I'm working on a series of sketches all utilizing the same set of core private headers, and having to make duplicate copies of them in every project because the compiler doesn't understand relative paths is a real pain in the neck.

Huh? But how would the feature requested by this issue help here? This issue requests that a copy of a library inside the sketch folder can be used just like a normal shared library. For your usecase, it sounds like you just need to make your "core private headers" into a library, install it into your sketchook "libraries" directory along with the other libraries, and then you can just include these from all sketches that need them?

@alandsidel
Copy link

I want to add a +1 here too. I'm working on a series of sketches all utilizing the same set of core private headers, and having to make duplicate copies of them in every project because the compiler doesn't understand relative paths is a real pain in the neck.

Huh? But how would the feature requested by this issue help here? This issue requests that a copy of a library inside the sketch folder can be used just like a normal shared library. For your usecase, it sounds like you just need to make your "core private headers" into a library, install it into your sketchook "libraries" directory along with the other libraries, and then you can just include these from all sketches that need them?

You're right, the issue as it stands wouldn't be helpful. I was referring to per1234's message a few up where he mentions e.g. #include "../../foo/src/foo.h" which itself wouldn't work either.

My personal goal is to get away from using the Arduino IDE and to VS Code for a variety of reasons, so adding to some IDE specific library manager doesn't seem like a good path forward. The forums have indicated to me that including with relative paths simply doesn't work in the Arduino IDE because the files are copied somewhere else prior to compilation, and it seems like the VS Code arduino plugin seems to do the same thing (I guess?) before calling avr-gcc, or however it enters the ecosystem's toolchain.

@per1234
Copy link
Contributor

per1234 commented Aug 9, 2020

I was referring to per1234's message a few up where he mentions e.g. #include "../../foo/src/foo.h" which itself wouldn't work either.

I should clarify that when I mentioned #include "../../foo/src/foo.h" in my previous reply, that was referring to the #include directives in a bundled library that has a dependency on another bundled library, not to #include directives in a sketch.

I'll provide a more detailed explanation:

Let's say you have a sketch with a structure like this:

MySketch
|_ MySketch.ino
|_ src
     |_ bar
     |    |_library.properties
     |    |_src
     |        |_ bar.h
     |_ foo
          |_ library.properties
          |_ src
               |_ foo.h

With a normal library installation to the libraries subfolder of the sketchbook, if the "bar" library has a dependency on the "foo" library, it can just use #include <foo.h> and the Arduino library dependency resolution process automatically finds the "foo" library. However, if I want to bundle the "bar" and "foo" libraries with my sketch, as in my example structure above, then I must edit the "bar" library and modify #include <foo.h> to provide the relative path to foo.h: #include "../../foo/src/foo.h".

The forums have indicated to me...

Here's the forum thread @alandsidel mentioned: https://forum.arduino.cc/index.php?topic=699636

@Steve132

This comment was marked as duplicate.

@alandsidel

This comment was marked as off-topic.

@pwyq

This comment was marked as duplicate.

@forkineye

This comment was marked as duplicate.

@alandsidel

This comment was marked as off-topic.

@forkineye

This comment was marked as off-topic.

@alandsidel

This comment was marked as off-topic.

@forkineye

This comment was marked as off-topic.

@Steve132
Copy link

I implemented this feature and made a pull-request arduino/Arduino#11110

@per1234 per1234 changed the title No support for libraries in sketch folder Support bundling libraries with a sketch Apr 5, 2021
@per1234 per1234 transferred this issue from arduino/Arduino Apr 5, 2021
@per1234 per1234 added topic: core type: enhancement Proposed improvement labels Apr 5, 2021
@per1234 per1234 added the topic: code Related to content of the project itself label Jan 21, 2023
@Levi-Lesches

This comment was marked as duplicate.

@per1234 per1234 linked a pull request Feb 24, 2024 that will close this issue
6 tasks
@arduino arduino locked as too heated and limited conversation to collaborators May 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
topic: code Related to content of the project itself type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging a pull request may close this issue.