recommend explicit using Foo: Foo, ... in package code (was: "using considered harmful")

[KristofferC]

I feel we are heading up against a "using crisis" where any new feature that is implemented by exporting a new name (either in Base or a package) becomes a breaking change. This is already happening (JuliaGPU/CUDA.jl#1097, JuliaWeb/HTTP.jl#745) and as projects get bigger and more names are exported, the likelihood of this rapidly increases.

The flaw in using Foo is fundamental in that you cannot lexically see where a name comes from so when two packages export the same name, you are screwed. Any code that relies on using Foo and then using an exported name from Foo is vulnerable to another dependency exporting the same name.
Therefore, I think we should start to strongly discourage the use of using Foo and only recommend using Foo for ephemeral work (e.g. REPL work).

[DilumAluthge]

Should we also say that you can just do import Foo instead of using Foo: Foo?

[mbauman]

What's the status of tooling support here? I'd say that's required before we make any such proclamation.

[ararslan]

Good point about tooling. Something very nice that Racket's IDE DrRacket does is that it shows you the source of a binding when you hover over it. That very nicely mitigates any confusion when reading code over the source of a particular binding.

Also note that my suggested change to the verbiage makes this more informative and less prescriptive, which could be worthwhile regardless.

[DilumAluthge]

less prescriptive

Personally, I think we should be more prescriptive. As Kristoffer points out, this kind of use of using is actively breaking packages.

[ararslan]

I think it should be left up to code authors how they want to write their code. We can provide information that may inform their decision, but saying "Julia provides a way for you to do this but don't do it lul" isn't helpful. People find and fix breakages on the (in my experience, rare) occasions that they occur, users can qualify bindings or do explicit loads if they choose.

When Base adds a new export, we should probably run PkgEval to see what effect it has. It would also be nice if packages with many dependants had reverse testing on new tags. That way package authors can be better equipped to work together to provide a good experience for users.

[tkf]

People (including me) don't read documentation carefully always. Starting with a concise recommendation followed by reasoning seems to be the best practical strategy. If they disagree with the recommendation and understand the reasoning, it is of course "left up to code authors how they want to write their code."

One missing reasoning step in the current version is that, adding features ¹ is not considered breaking and can be done without incrementing the major version according to semver. So, if you only bound the major version and not minor version, you simply cannot ² use using Foo; (strictly speaking). This gives Julia programmers another strategy: they can use using Foo; and bound the minor version of all dependencies.

Footnotes

1. I'm implicitly assuming exporting a new name is considered a feature addition. I believe this is the common understanding but I don't know if that's explicitly noted. ↩

2. One exception is using Foo; in a baremodule which is still semver-compatible. ↩

[jlapeyre]

In many situations, I favor using Foo: Foo or import Foo and then Foo.a. If a is a long identifier that is heavily used, then maybe using Foo: long_identifier. You might want to mention it as an option for solving the problem. There are arguments for and against fully qualifying names that I won't go into here.

In any case, I haven't seen a good defense for using Foo in library code.

The admonition seems like it belongs in some kind of style guide. And tooling might include an option in linters, with an option at some level to error on warnings. I don't see why it needs a certain level of tooling support before the recommendation is published. But, without tooling, I suspect adoption will remain weak. I'm thinking of the ways PEPs and pylint work in Python projects and clippy in rust. I not suggesting the big administrative burden of the whole PEP system, just that starting to adopt styles and tooling to handle complexity as the amount of code increases is wise.

[domluna]

It could be an option for JuliaFormatter, there was talk about this in a slack thread. The thing about that is I think the only people who would turn it on are people who already know about the pitfalls using using in such a manner. So it would be preaching to the choir in a sense.

[tkf]

Arguably, there's nothing new in this PR. It has been (more or less) possible to derive this from the semver specification.

While discussion on tooling is important on its own, I believe a more fundamental aspect is the interaction of semver and Julia's package/module system. Making it crystal clear even for those who haven't read the semver specification will help to reduce "controversies" of choosing the behavior of such tools. (I'm sorry for RTFM'ing. But I feel that this is an important point here.)

[johnnychen94]

I would instead recommend the other side: be conservative on what you should export and think about the naming ambiguity.

For instance, exporting a width trait from any package should be discouraged IMHO because 1) this trait is mostly for developers and not for users or convenient usage 2) this name is so common that nobody except for Base should declare the ownership on it.

From a maintenance perspective, exporting a function at will also makes it very difficult to deprecate and remove the symbol.

[timholy]

I think this is perhaps stronger advice than required. How about pointing out the problem and proposing a way to address it without prescribing that "you shouldn't do this." Personally I'm OK with a rare breakage if I get to write nicer code internally in my packages.

[KristofferC]

What's the status of tooling support here? I'd say that's required before we make any such proclamation.

Out of curiosity, why? The presence or absence of tooling doesn't change any of the facts.
Sure, having tooling to convert a using Foo into explicit names would make it easier to move away from naked using but I don't see how it changes anything concrete.

[KristofferC]

I think it should be left up to code authors how they want to write their code. We can provide information that may inform their decision, but saying "Julia provides a way for you to do this but don't do it lul" isn't helpful.

Sure, this is why this is only a recommendation and this section just states the recommendation and the reason for it (exporting new names becomes a breaking change).

[timholy]

But the language is still quite prescriptive: "X is not recommended, do Y instead" vs "X can be messed up by Z, you can prevent this by doing Y". Subtle difference, but one that I think affects perception.

[KristofferC]

Personally I'm OK with a rare breakage if I get to write nicer code internally in my packages.

Just to give you an example of what this leads to (JuliaWeb/HTTP.jl#745 (review)). HTTP.jl now does type piracy on closewrite(::Any) because they consider it breaking to not allow using HTTP; ...; closewrite(stream) to work. So it isn't really a "rare breakage" that gets fixed and everyone is happy but long-lasting bad situations from these choices. And while you might fix the breakage on the latest version, do you also backport that to all your other semver incompatible versions? Otherwise, people who do not want to upgrade to a new breaking version of your package are stuck with their old version not working anymore.

And with "nicer" code you mean to avoid the annoyance of having to spell out where your names come from. While you might be smart enough to remember exactly where all the names come from, I am not, so trying to look up where certain names come from is one of the main annoyances I have while reading code. Explicit mentioning where names come from would be a nice help to future contributors who don't have the namespace of all the dependencies in their head.

[tkf]

@ararslan @mbauman @timholy If this PR is rephrased to a mere weak recommendation, shouldn't the use of semver in packages become a weak recommendation, for consistency? See my above comments for the context.

(It's not a rhetorical question. It sounds like an option.)

[KristofferC]

One missing reasoning step in the current version is that, adding features [^1] is not considered breaking and can be done without incrementing the major version according to semver. So, if you only bound the major version and not minor version, you simply cannot [^2] use using Foo; (strictly speaking).

Yes, the "axiom" here is that assuming everyone following semver and declares compatibility correctly, then we want to encourage coding in a way that keeps the code after a package update. That's why we discourage the use of type piracy, we discourage using internals of packages/Base. And as you say, an assumption is that adding a new export is not considered a breaking change (which I think is quite uncontroversial, looking at how Base does it). The conclusion from that is that since using Foo is incompatible with the two previous points, it should be discouraged.

[DilumAluthge]

Any particular reason for not backporting this? I imagine that people using the LTS will be reading the 1.6 docs, and it would be good for them to see this recommendation.

[DilumAluthge]

What's the status of tooling support here? I'd say that's required before we make any such proclamation.

Out of curiosity, why? The presence or absence of tooling doesn't change any of the facts.
Sure, having tooling to convert a using Foo into explicit names would make it easier to move away from naked using but I don't see how it changes anything concrete.

I agree that having tooling is not a pre-requisite for merging this PR.

[mbauman]

I'd be okay with a descriptive warning without there being tooling support.

I don't like having a prescriptive admonition without tooling support. With tooling support, I'm 100% for it (hence my conflicting thumbs). Perhaps I'm in the minority, but I don't think this is a minor point. To make a (poor) analogy, some new governmental regulations are accompanied by tax credits/grants/assistance to offset their implementation cost. We should make it easy for folks to address. Without that, it's just words that core devs will point at to blame packages for breaking.

[mbauman]

Similarly, I think a descriptive warning (like Alex's suggestion) could be back ported to 1.6/1.7, but a prescriptive one should not be.

[tkf]

To make a (poor) analogy, some new governmental regulations are accompanied by tax credits/grants/assistance to offset their implementation cost.

The point is that the "regulation" (= semver) already exists.

(I don't believe tooling should be a prerequisite but I'd also point out creating one is straightforward thanks to JuliaFormatter.jl. Here's my POC I experimented a while ago for automatically generating using Foo: f style imports: https://github.com/tkf/AutoImports.jl)

[MasonProtter]

Looks like someone just needs to commit @ararslan 's suggestion and then merge

[IanButterworth]

I'm marking for merge, but a review approval wouldn't hurt

[MasonProtter]

@ararslan can you remove your block on merging?

Suggested changes were made