Add support for path dependencies in crates.io uploads

[GuillaumeGomez]

Hi,

I have the following error when I run cargo package:

> cargo publish
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Packaging glib v0.0.1 (file:///home/imperio/rust/glib)
   Verifying glib v0.0.1 (file:///home/imperio/rust/glib)
    Updating registry `https://github.com/rust-lang/crates.io-index`
failed to verify package tarball

Caused by:
  no matching package named `glib-sys` found (required by `glib`)
location searched: registry https://github.com/rust-lang/crates.io-index
version required: ^0.1.0

The problem is that I have this sublib path in the cargo.toml file:

[dependencies.glib-sys]
path = "./glib-sys"
version = "0.1.0"

No problem with the cargo build however. So I changed the line to:

[dependencies.glib-sys]
git = "https://github.com/rust-gnome/glib"
version = "0.1.0"

(which is also the link to the library).

But when I run cargo publish:

all dependencies must come from the same source.
dependency `glib-sys` comes from https://github.com/rust-gnome/glib instead

When reading the tutorial to publish library, it says that path is supported. However it seems not. Is it a bug ?

PS: Here is the link to the glib repository.

[GuillaumeGomez]

So like Dabo proposed, I think it would be nice to add path dependencies support for crates.io uploads. In the case of all rust-gnome repositories, it would be very helpful.

What do you think of it ?

[alexcrichton]

Right now you'll have to publish each component of a crate independently (e.g. publish glib-sys first), and otherwise I think this falls under #1169 (where it would publish both at once), so I'm going to close in favor of that. Thanks for the issue though!

[nwin]

But #1169 is a totally different issue. This is (as far as I understand) not about uploading a sub-crate but to include sub-crates into a package.

I do not see how path dependencies should be harmful as long as they get uploaded to crates.io like any other file in the package.

[real-or-random]

I agree, #1169 is different.

@alexcrichton Can you reconsider re-opening this issue? It may not be a good idea in the end but it's worth discussing.

[alexcrichton]

@real-or-random what do you see the outcome of this issue being? How would this get resolved?

[crumblingstatue]

The idea is to allow packages with path dependencies to be uploaded to crates.io. This could be achieved by simply packaging the path dependencies along with the main crate.

A possible motivation for this could be splitting up a project into separate crates for better compilation performance/etc., or using a private fork of some other library temporarily.

[alexcrichton]

Such a feature is basically entirely incompatible with all of Cargo right now, and would require a significant rewrite from the ground up to support unfortunately.

[real-or-random]

Looking at projects such as imag, which is spread over > 30 crates, I had the feeling that packaging path dependencies will be a very useful feature and that #1169 is the wrong reason to close it. I don't have any specific outcome in mind.

I'm new to rust and I'm probably the wrong person to judge the effort to build this but even some notion of a group (just on crates.io, independently of cargo) will help in certain scenarios (this will be closer to #1169 then).

On the other hand, there was a deliberate decision to use a flat namespace, and it's possible to create "meta-crates" which do nothing but depending on other crates if you want groups.

[alexcrichton]

It sounds like the requested feature, in terms of cargo internals, would be to support packages that have multiple library crates. That feature is quite large and is explicitly not supported by Cargo today, but it sounds like it would solve the use case here I believe. If you'd like to open a dedicated issue for that, please feel free to do so!

[gnzlbg]

@alexcrichton we can't release stdsimd on crates.io anymore without publishing stdsimd-test :/

[jakwings]

@alexcrichton Hi, my idea is that sub crates with package.publish = false should be uploaded.

[jonhoo]

This is hitting us with noria as well. Specifically, I'd like to publish the sub-crate noria-server, but it internally contains three relatively large (private) sub-crates (dataflow, mir, and common). While it would be possible to turn all those sub-crates into modules, it makes compilation painfully slow.

[Timmmm]

This just bit me. I think it should definitely be fixed, though in the mean time it could at least be made less annoying. First, the error message I got is not great:

error: all dependencies must have a version specified when publishing.
dependency `my_internal_crate` does not specify a version
Note: The published dependency will use the version from crates.io,
the `path` specification will be removed from the dependency declaration.

Confusing - why is it complaining about versions? I only got a clue that this was the problem from the very last line. I suggest something like this:

error: all dependencies must be specified by version when publishing but
dependency `my_internal_crate` is only specified by path.
Note: If you specify both `path` and `version` for a dependency then
`path` will be used locally but `version` will be used when published.

Second none of the documentation on path mentions this pretty big limitation!

I would add a comment in this section saying something vaguely like

Note: path is ignored when publishing on crates.io. Crates.io does not support internal crate dependencies - all of your path dependencies must be published as separate crates and referenced by version.

On this page it does at least tell you:

If you publish the crates in the workspace to crates.io, each crate in the workspace will need to be published separately. The cargo publish command does not have an --all flag or a -p flag, so you must change to each crate’s directory and run cargo publish on each crate in the workspace to publish the crates.

But it's right at the bottom! I'd suggest moving it higher or making it into a yellow warning box (assuming you can do that).

[notdanilo]

I also think this issue should be reconsidered and fixed. We want to be able to separate the project in different crates to improve the project organization, but we don't want them to get published.

[Timmmm]

One way this could at least be improved is to have an "internal" flag in Cargo.toml to prevent the crates showing up on crates.io.

[AlbertMarashi]

+1 for this feature

[acheroncrypto]

It is really painful to publish all local dependencies after 7 years.

[sam0x17]

This really needs to be addressed. This makes it so hard to maintain proc macro crates

[kang-sw]

+1 here. I wish I could use a forked crate without publishing it to crates.io and cluttering its namespace. I just want to include it in my crate until the pull request gets approved. :|

[Rafaeltheraven]

+1 for this issue making proc macro crates even more of a nightmare to deal with than they already are

[AlbertMarashi]

I created a crate for this:

1. create an armory.toml in your Cargo Workspace with a version = "your_current_version" field
2. Run cargo install cargo-armory
3. Run cargo armory -> select whether you want to release as a patch, minor or major

https://docs.rs/crate/cargo-armory/0.4.1