Specify dependencies for binary distributed frameworks

[ghost]

Hello,

For my company, I'm taking care of building and distributing a framework.
This framework contains proprietary code, and therefore cannot be distributed through git/github.

The archive is building just fine, and is correctly uploaded to our server. The clients can integrate the framework in their projects.

Unfortunately, our framework has a few dependencies(see Cartfile below), and currently the only way for ours client to integrate our framework is to make them add our dependencies to their Cartfile.

Is there a way to specify dependencies for frameworks distributed with the binary scheme?

I suppose it's not that big of an issue because they would have to add our dependencies to Linked Frameworks and Libraries and /usr/local/bin/carthage copy-frameworks anyway, but it would be a bit cleaner to make Carthage handle the dependencies for binary frameworks.

Could we add the framework's Cartfile to the archive, and use it to resolve the final app's dependencies?
Or have I missed something?

• carthage version: 0.22.0
• xcodebuild -version: Xcode 8.3.2 Build version 8E200
• Are you using --no-build? No
• Are you using --no-use-binaries? No
• Are you using --use-submodules? No
• Are you using --cache-builds? Yes

Cartfile

github "DaveWoodCom/XCGLogger" ~> 5.0.1
github "marmelroy/Zip"
github "Alamofire/Alamofire" ~> 4.4

PS: see http://stackoverflow.com/questions/42409320/is-there-anyway-to-specify-library-dependencies-when-using-binary-origin for someone with the same issue

[mdiep]

There's not currently a solution for this, no.

[wlisac]

+1. Supporting this would be great.

[tmspzz]

This issue #2294 discusses improvements to the Cartfile and a possible new format where what you want should be possible

[kuchmiyalex]

+1 also gonna need this to make integration of proprietary sdk for clients easier

[tmspzz]

@kuchmiyalex feel free to work on the issue.

[mdiep]

This shouldn't be too difficult. A great first step would be to open an issue proposing a change to the JSON format to support this.

[JonasVautherin]

Following @mdiep's suggestion to extend the JSON format defined here and "JSON-ifying" the Cartfile definition, I would see it like this:

{
    "1.0.0": {
        "archive": "https://my.domain.com/release/1.0.0/framework.zip"
    },
    "1.1.0": {
        "archive": "https://my.domain.com/release/1.1.0/framework.zip",
        "dependencies": [
            {
                "type": "binary",
                "link": "https://my.domain.com/release/MyFramework.json",
                "tag": "~> 2.3"
            },
            {
                "type": "git",
                "link": "file:///directory/to/project",
                "tag": "branch"
            },
            {
                "type": "github",
                "link": "jspahrsummers/xcconfigs"
            }
        ]
    }
}

Where the dependencies exactly match the Cartfile, i.e. all the examples would apply.

That said, all that is needed is for a binary-only dependency to expose its own Cartfile. So my question would be the following: why not having the Cartfile in the archive (i.e. framework.zip would contain a Cartfile) and resolve that when fetching the binary? This way the dependencies would not need to be duplicated in the json file.

[mdiep]

That's definitely possible. The only issue is that you need to download and unarchive a zip to find its dependencies. That's potentially much slower and would require more disk space.

[JonasVautherin]

Thanks for the answer! What do you think about the JSON proposition?

Regarding the Cartfile solution, would you mind elaborating on that? My intuition is that if my project depends on a framework, then Carthage will eventually download this framework. Why is that wrong? I don't get the difference between knowing the dependency graph in advance and downloading the dependencies "sequentially"... And I don't get why it would use more disk space either 🤔.

[mdiep]

Say there are 50 releases and your Cartfile is only compatible with the earliest one.

If you include the dependencies in the JSON, you can figure that out before downloading and only download one .zip.

If you have a Cartfile inside the .zip, then you need to download and unarchive all 50 to find one that's compatible. We cache binary downloads, so those will also stick around indefinitely.

[JonasVautherin]

I am not sure I follow, but I can believe you if it takes you too much time to explain (and then we can discuss the JSON solution). But maybe I was not clear.

Say there are 50 releases and your Cartfile is only compatible with the earliest one.

This is solved by the current JSON file, and I agree it is necessary. My point is that to solve this issue, maybe the JSON file doesn't need to be affected at all. Carthage should just follow the Cartfiles it finds in the binaries it downloads (and it will not download 50 binaries, because there is the JSON file that says which archive you want exactly).

Let's try with an example:

Say my Cartfile says something like:

# A binary only framework
binary "https://my.domain.com/release/framework.json" ~> 2.3

Carthage gets MyFramework.json, that looks like:

{
    "2.0": "https://my.domain.com/release/2.0/framework.zip",
    "2.3": "https://my.domain.com/release/2.3/framework.zip"
}

Carthage therefore downloads <...>/2.3/framework.zip and unarchives it. In the archive, there are two items:

• framework.framework // The actual framework I just downloaded
• Cartfile // The Cartfile of the framework I just downloaded

Carthage therefore reads this newly-discovered Cartfile, and continues until there are no more dependencies. That's exactly what Carthage does with open source repos, right? It follows the Cartfiles. Just that in this case everytime a Cartfile mentions a JSON file, then the binary archive needs to be downloaded before its own Cartfile can be read.

Does that make sense?

[tmspzz]

Carthage therefore downloads <...>/2.3/framework.zip and unarchives it.

Not quite. What it downloads is the minimum compatible version. Which of the two depends on the dependency resolution happening when the Cartfile.resolved is created. To create the resolved you would have to download all the zips.

[JonasVautherin]

Right. So it is actually the case that Carthage needs the global resolution before it can start downloading the archives.

Then it probably means that the dependencies need to be exposed in the JSON (as first suggested by @mdiep), at the cost of having to maintain both the JSON and the Cartfile when providing a binary-only framework (which, I believe, is acceptable).

What do you think about the following extension to the JSON format? I don't see many other ways to do it:

{
    "1.0.0": {
        "archive": "https://my.domain.com/release/1.0.0/framework.zip"
    },
    "1.1.0": {
        "archive": "https://my.domain.com/release/1.1.0/framework.zip",
        "dependencies": [
            {
                "type": "binary",
                "link": "https://my.domain.com/release/MyFramework.json",
                "tag": "~> 2.3"
            },
            {
                "type": "git",
                "link": "file:///directory/to/project",
                "tag": "branch"
            },
            {
                "type": "github",
                "link": "jspahrsummers/xcconfigs"
            }
        ]
    }
}

[tmspzz]

I don't thing "tag" and "link" are the appropriate names for those keys.

Instead of "link", maybe location ? Instead of "tag" maybe constraint ?

I very much like the idea of converting the Cartfile toJSON. See also #2294 for other ideas.

[ikesyo]

Just putting the current Cartfile syntax may be sufficient for now I think:

"dependencies": [
    "binary \"https://my.domain.com/release/MyFramework.json\" ~> 2.3",
    "github \"jspahrsummers/xcconfigs\""
]

I very much like the idea of converting the Cartfile toJSON.

I'm not for that, JSON configuration files are less readable (at least for me) and we can't write comments in JSON.

[tmspzz]

less readable (at least for me) and we can't write comments in JSON.

In #2294 I suggested yaml that can help in both.

[JonasVautherin]

@blender @ikesyo: from your answers, can I assume that we all agree on the idea of extending the binary project specification to add dependencies?

If yes, would you happen to have an intuition on how difficult it would be to change that and/or where one should start in the codebase?

Just putting the current Cartfile syntax may be sufficient for now I think

@ikesyo: It's not a strong request on my side, but I really don't like that. Having to escape a double quote there looks like a hack to me. I would vote for json or yaml. Because json is already deployed (again, talking about the binary project specification), and because anyway this file is more than only a Cartfile (it is actually the concatenation of all the Cartfiles of the different releases), I believe it makes sense to stay with json. But that's definitely negociable.

Instead of "link", maybe location ? Instead of "tag" maybe constraint ?

@blender: fine for me :-)

[JonasVautherin]

I am also realizing that because Swift is not ABI-compatible, it makes the binary-only system more complicated.

Should a field be added to the json file to take that into account, or is it the problem of the developer providing the framework?

[hlineholm]

Anyone working on this?