Guidance around reexporting

[sanmai-NL]

Guidance around reexporting has been removed. I propose to pick up this issue again, with high priority, since it has a potentially large impact on, at least, the open source Rust ecosystem, as the common practice around this issue greatly influences:

• API surface of crates that have public dependencies (a great number of them); as well as
• the degree of coupling between crates where C depends on B, which depends on A as a public dependency, where crates A, B, and C are all part of the open source ecosystem.

There are number of questions to resolve around this.

• The interaction between public dependencies and reexports.
• pub use vs. pub extern. When to use the one over the other. Whether to ever use them both for the same items.

Draft best practice

If your crate has a public dependency, then the corresponding crate should be reexported by your crate (via pub extern). First of all, why consider reexporting anything? The main motivation is to help consumers C of your API in crate B avoid having to depend on your own dependencies, e.g. A, in order to work efficiently with your crate B. Would you have failed to reexport, then:

• It would be inconvenient as the extra dependencies for your consumers have to be specified.
• It would be also a source of instability for the consumer, since it requires work on the part of the consumer to find out what version of the dependency A is compatible with your crate B (as well as consumer’s crate C).

It is better to use pub extern and preserve crate A’s structure directly in your crate B’s root rather than selectively exporting items from A via pub use at an idiosyncratic path:

• It is hard to tell what items from dependency crates are practical to offer for API consumers of your crate.
• Furthermore, it clutters the namespace of your own crate to pick up items from other crates, if you do not preserve their original module structure, e.g. via pub use. You risk not being able to name items as you prefer within your own crate at some point in the evolutions of your crate and the dependency crate.

[KodrAus]

@sanmai-NL So you're specifically talking about re-exporting foreign items from dependencies rather than re-exporting local items in the same crate under a more convenient path.

Personally, I think creating public dependencies by re-exporting foreign items should be at least mildly discouraged since in my experience it creates a future maintenance burden that outweighs the convenience of the re-export in the first place. As an example, I recently had to back out of a pub use in env_logger which turned out to be a fairly complex change because the API I was effectively re-exporting was much larger than just the items I re-exported (because of trait impls).

The hyper + http example is probably more of an exception, since @seanmonstar has some control over the APIs of both those crates and http is specially designed as a library of public interchange types.

In cases where you do determine pub use for foreign items is an appropriate strategy then re-exporting the crate seems like a fair thing to do, even if you only really need a small part of it. I think it's better to wear the boilerplate of wrapping foreign items whenever possible though.

[BurntSushi]

@KodrAus Hmmm, I think this issue is starting with the assumption that you have a public dependency for one reason or another, and then trying to answer a question about re-exports given that you already have a public dependency. I think the termcolor thing is an example where you could avoid the public dependency in the first place, so is probably in a different class of API design than what this issue is trying to tackle (if I've understood correctly).

[KodrAus]

@BurntSushi Ah I was approaching this from the angle of deciding whether or not to create a public dependency by re-exporting foreign items, rather than whether you should re-export the crate for your existing public dependencies. So are we looking at whether you should re-export the crate dependency in the case where it becomes a part of your public API in any way? Like:

// pub use dependency; <- should we do this in the crate root?
 
pub fn pub_fn(pub_arg: dependency::Arg) { ... }

// pub use dependency; <- should we do this in the crate root?
 
pub struct PubStruct { ... }

impl dependency::Trait for PubStruct { ... }

[sanmai-NL]

My proposal is to encourage re-exporting the crate for your existing public dependencies.
Also, I want to suggest not using pub use but pub extern crate, until the dust is settled around rust-lang/rust#44660. pub (use) is too flexible and does not work within nested use statements.

[seanmonstar]

I think there is a bit of "it depends", even in this guideline (which may be why it was removed). For instance, these are cases of having a public dependency where I'm not sure it makes sense to re-export the crate:

• If you implement Serialize, should pub extern crate serde?
• If you implement Future, should you pub extern crate futures?
• Things like hyper-tls, hyper-rustls, etc make connectors for hyper, should they pub extern crate hyper?

[sanmai-NL]

I think yes in all cases. Fundamentally, this best practice is about predictability, not about judging how public dependencies C are going to be used by consumers A via crate B. Indeed, that’s what makes this best practice easy to adopt: there’s no subjectivity. Also, though somewhat orthogonal, we can write a few words about how reexporting a public dependency materially increases API surface of your crate, but since that part of the API is of a different nature then the ‘intrinsic API’ (for lack of a better, simpler word?), this does not matter. Different API stability guarantees may apply to the ‘extrinsic API’ (reexported public dependency crates).

It should be possible to add a lint that checks whether all public dependency crates are reeexported. Meaning all dependency crates with any item making up the extrinsic API of the linted crate. If so, this best practice would lend itself to automated checking.

[softprops]

I see this mostly as a user experience problem than anything.

One approach would be reexporting as mentioned but what if you had two dependencies which depended on futures but depended on different versions of futures. Would mixing foo::futures::Future and bar::futures::Future be an improvement or cause more UX problems?

Has anyone explored the idea of cargos knowledge of transient dependencies filling in ( implicitly ) these dependencies as first class dependencies of application crates. I know there's already work underway with the 2018 edition to remove extern crate in source files. If this were the case, having cargo fill these in would be a nicer ux for application crates. If hyper depends on futures exposing it in public interfaces then an application depending on hyper may also depend on futures without an application crate needing to spell that out manually in thier cargo.toml.

[sanmai-NL]

@softprops, as before you’d have to careful what you import and how. Before, you could still import crates at incompatible versions.

[dekellum]

Doesn't the practice of consuming re-exports instead of direct imports tend to reinforce or increase the likelihood of multiple versions of the same dependency within the complete project dependency graph? Duplicates are always better avoided when possible, right? Or is the proposal here to always give the opportunity by re-exporting, with the hope that consumers are thoughtful about when to use them?

For a practical example of how I arrived at this theory: I started off with a "hyper 0.10 application" for which, I have since come to realize, is better understood as a "tokio (reform) 0.1.x" application with an additional hyper (now 0.12) dependency. Should that subtle/conceptual distinction influence the choice to use re-exports over direct dependencies in an application? I think it always will be influential in practice.

[sanmai-NL]

@dekellum:
Are they to be avoided, IMO yes. I believe this increases binary sizes. You could take the position that this concern is separate and should be handled by a delivery QA control, e.g. running cargo tree -d during testing to detect duplicate dependencies and handling them as appropriate, depending on policy.

Or is the proposal here to always give the opportunity by re-exporting, with the hope that consumers are thoughtful about when to use them?

Yes. 🙂

But let me get this straight: Cargo resolves dependencies based on constraints. If there is a single compatible version of, say, hyper in the dependency graph, there won’t be a problem. If not, then using a dependency via path A at version x is only interchangeable with the same crate via path B at version y, if these versions are API compatible. That shouldn’t occur with thoughtful use of and conformance to SemVer versioning, right? There can be an issue still, which would be obscured if only a proper subset of the API is used and the incompatibility is in the unused part.

Do we have the same understanding here?

[dekellum]

Thanks @sanmai-NL, I think my original concern regarding duplicates exists but wasn't well stated. More on that later.

Back to your proposal, I would also prefer pub extern crate (while its a thing?) over deep linking/renaming via pub use.

By comparison, pub use can be a cognitive burden and annoyance when attempting to learn multiple crates simultaneously. For example if you are browsing the docs and land at hyper::rt::Stream there is no way of discerning that this doc is lifted from the futures crate and no obvious way to navigate, for example, to the futures::stream module docs and its list of other types and helper functions. Conversely, if you happen to click from the hyper::rt::Stream doc to a type that isn't pub use relocated by hyper, for example IntoFuture, then you unexpectedly (without warning) land in the futures crate docs and might not understand how you arrived there. The rustdoc output could be improved for pub use, but I think the docs isn't the only place the cognitive burden is felt.

[jsgf]

I think the key issue is that if I have crate A which uses crate B as part of its public API, and I have crate C trying to consume A's API, then when I construct C's Cargo.toml, I need to have exactly the same version constraint for B that A is using. This is pretty awkward and fragile, and it means that even a minor change of A's version constraint is potentially a breaking change for C.

On the other hand, if A fully re-exports B, then C can directly consume A's version of B without having to worry about version drift.

The question of whether there's another version of B floating around is secondary; maybe there is if D also uses a different version of B. That's not desirable, but out of scope for this particular issue.

[grncdr]

then when I construct B's Cargo.toml, I need to have exactly the same version constraint for B that A is using

Did you mean "when I construct C's Cargo.toml" in the above?

[jsgf]

@grncdr Yep.

[Kixunil]

Had this exact same idea, even opened an issue at clippy

I think an exception could be made if a type from a crate implements a trait that is fundamentally useless on its own. E.g. you probably want an executor for a Future which most likely will be in another crate and you don't even need to touch the trait directly. Same with serde::Serialize (you can e.g. pass the type directly to serde_json::to_string() and there's a good chance you will not name the trait).

But I still prefer to reexport the crate in that case. One could wish to implement the logic for trait without other crates and the change required is minimal.

Also note one important thing: once a type from another crate appears in the public interface then reexporting it will not add any new requirements on backwards compatibility. There's no way one could backwards-compatibly reimplement the type later because the fact that the type is the same as one from another crate becomes a part of the API! This is a good reason for people to think twice before adding an external type to public interface in the first place. Once it's there, the crate should be reexported.