[SR-145] Need a way to add include paths in modulemap

[swift-ci]

Previous ID	SR-145
Radar	None
Original Reporter	terhechte (JIRA User)
Type	Bug
Status	Resolved
Resolution	Done

Environment

Ubuntu Linux 15.04 with GTK+3

Additional Detail from JIRA

Votes	8
Component/s	Compiler, Package Manager
Labels	Bug
Assignee	None
Priority	Medium

md5: 0f2c4d0bda8df613a9af477cdc681d26

is duplicated by:

• SR-179 Including Gtk header (gtk/gtk.h) in a module - can't find gdk/gdk.h

Issue Description:

I tried to get GTK running with Swift. To run this, a Linux machine with a working GTK3
installation is needed (could also work with OSX && brew install gtk+3, however I did not try that);
i.e.
`sudo apt-get install libgtk-3-dev`
Initially I tried with GTK but since GTK has dependencies to a lot of other things, I tried something
simpler, i.e. in this case GDK which has only dependencies to X11.

// First try
module CGTK [system] {
umbrella header "/usr/include/gtk-3.0/gdk/gdk.h"
link "gdk-3"
export *
}

Error:
Cloning Packages/CGTK
Compiling Swift Module 'example' (1 sources)
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/usr/include/gtk-3.0/gdk/gdk.h"
^
/usr/include/gtk-3.0/gdk/gdk.h:30:10: error: 'gdk/gdkconfig.h' file not found
#include <gdk/gdkconfig.h>

More here:
https://gist.github.com/terhechte/c48fff0eb7e581d676b2

[swift-ci]

Comment by Tomáš Linhart (JIRA)

I have also encountered same problem on Mac OS X. I have created test binding for Gtk+, it works nicely with Xcode because I can define header search path for all headers but unfortunately, it is not working with spm.

I guess, this problem will pop out with more libraries that have a lot of dependencies. I have noticed spm is passing into swiftc -I /usr/local/include. Maybe if we were able to pass anything we want, it would solve the problem? Maybe be able to define it in Package.swift.

[swift-ci]

Comment by Dan Appel (JIRA)

I'm having the same issue expect with the MongoDB C Driver:

$ swift build
Cloning Packages/CMongoC
Compiling Swift Module 'SwiftMongoDB' (12 sources)
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/usr/local/include/libmongoc-1.0/mongoc.h"
        ^
/usr/local/include/libmongoc-1.0/mongoc.h:22:10: error: 'bson.h' file not found
#include <bson.h>
         ^
/Users/dan/Developer/projects/SwiftMongoDB/Sources/Declarations.swift:9:8: error: could not build Objective-C module 'CMongoC'
import CMongoC
       ^

Adding the -I /usr/local/include/libbson-1.0 flag fixes the compiler error, but unless the rest of the invocations are executed manually the build process is not actually successful (l actually ran through these once and it did work).

Since the flags are stored inside generated yaml files, the next obvious step was to add the flag there, but unfortunately that doesn't work because the yaml files are regenerated from scratch each time swift build is run. So, clearly that is not a working solution.

I think the two best options are (similar to the already suggested ones) to:

1. Support include paths in modulemaps
2. Support include paths in Package.swift

My guess is that supporting include paths in module maps will be more complex (since the package.swift solution would just involve passing an argument), but that's not up to me.

Edit: A third option would be to allow build arguments to come from environmental variables, which could be added as a workaround for now and then be built into the Package.swift later.

[swift-ci]

Comment by Y.E. Kwon (JIRA)

While I tried similar work, calling GLib-2.0 functions from swift code,

I had to modify swift-package-manager's source code to build code successfully,

line 211 of "Sources/dep/llbuild.swift" in swift-package-manager project,
there is a line of code appending header search path with "/usr/local/include"

// Swift doesn’t include /usr/local by default
args += ["-I", "/usr/local/include"]

I had added my own search path that I've got from `pkg-config --cflags glib-2.0'

// Swift doesn’t include /usr/local by default
args += ["-I", "/usr/local/include"]
args += ["-I", "/usr/include/glib-2.0"]
args += ["-I", "/usr/lib/x86_64-linux-gnu/glib-2.0/include"]

Then built swift-package-manager, got the executable binary of 'swift-build', replaced original distribution of swift with this version.

With this new version of 'swift-build' I could successfully compile the swift code that links to glib-2.0 library.

By this experiment, I guess/hope, swift-package-manager has to have the function that receive extra args from command line (or any other better way), as example; "swift build `pkg-config --cflags glib-2.0`"

[swift-ci]

Comment by Y.E. Kwon (JIRA)

There is discusstion related this issue, in Swift Package Manager PR #75.

The referred branch of PR could be a temporary work-around for a while until clang's modulemap has its own solution.

build SPM by "./Utilities/bootstrap"
replace newly built 3 binaries with originals
.build/.bootstrap/bin/swift-build
.build/.bootstrap/lib/swift/pm/PackageDescription.swiftmodule
.build/.bootstrap/lib/swift/pm/PackageDescription.so

[mxcl]

The problem here in fact is that you need a module map entry for GDK and BSON.

Module maps must depend on module maps for any dependencies they have.

If GDK and BSON have module maps too then the compiler will find their headers.

[swift-ci]

Comment by Dan Appel (JIRA)

How do you create a modulemap dependency? Obviously I have them set up as SPM dependencies and have tried setting up a nested modulemap. Sorry if this is clearly documented and I just missed it.

[mxcl]

dan (JIRA User) same as a normal dependency in Package.swift.

Anyway, I've looked into this more and in some circumstances you DO need include paths to be added. So I'm writing up a proposal for a full solution. Stay tuned.

[swift-ci]

Comment by Dan Appel (JIRA)

Alright. In my situation you most definitely do - I've tried every combination possible at this point.

Thanks for looking into it.

[drewcrawford]

I've PRed a workaround for this issue, #107. If that PR is merged, we can say

otherCompilerOptions:["-I", "IncludePath/"]

to work around this bug.

[swift-ci]

Comment by Jorge Luis Canizales Diaz (JIRA)

Hi @mxcl, I wondered if you got around to writing that proposal? I've run into this issue when trying to create modulemaps for Protobuf and gRPC.

Also, what's the channel to participate in Clang's modulemap design?

Thanks!

[mxcl]

Yes, and it was merged. Though if it exactly satisfies all needs will need you to test.

swiftlang/swift-package-manager#257

[swift-ci]

Comment by Jorge Luis Canizales Diaz (JIRA)

Ah, bummer, not in the modulemap itself? I'm bumping against this via Cocoapods, so Swift's package manager isn't involved in the workflow.

[mxcl]

Putting this sort of thing in the module map itself is not desired I think. The map describes required headers, not system-specific locations.

[swift-ci]

Comment by Jorge Luis Canizales Diaz (JIRA)

The specific need I'm facing would be to specify include paths relative to where the modulemap is (so OS-agnostic). This concept (of paths relative to where the modulemap is) is already used to specify headers and umbrella directories.

[mxcl]

You can use -Xlinker and -Xcc

[swift-ci]

Comment by Jorge Luis Canizales Diaz (JIRA)

Those are Swift PM flags, though. Cocoapods doesn't use it, and e.g. XCode's HEADERS_SEARCH_PATHS expects an absolute path (system-specific).

[mxcl]

I think you need to explain your situation more thoroughly. I'm guessing you are generating Xcode projects?

[abertelrud]

Sounds as if this issue predates the addition of system module packages (https://github.com/apple/swift-evolution/blob/master/proposals/0063-swiftpm-system-module-search-paths.md). Would that address some of the issues, if for example GTK were created as a system module package swift package init --type system-module?

[aciidgh]

This is resolved by addition of pkgConfig property as mentioned by @abertelrud