A discussion on helping downstream packaging

[mgorny]

A discussion aimed at explaining how downstream packaging works, what challenges are the packagers facing and how they can be helped.

---

📚 Documentation preview 📚: https://python-packaging-user-guide--1791.org.readthedocs.build/en/1791/

[ncoghlan]

Thanks for this!

Lots of comments inline, but many of them are just mechanical and stylistic things:

• suggested adding blank lines after the headings
• suggested changing the heading anchors to use-shorter-phrases-with-hyphens rather than repeating the titles
• suggested dropping most of the bold emphasis from the text (mostly dropped entirely, occasionally scaled back to just italics rather than bold)
• suggested rewording for anything containing the pronouns "you" or "your" (it's really hard to make second person pronouns sound non-confrontational in English, so it's simpler to avoid them entirely)
• suggested rewording anything containing the pronouns "we" or "us" or "I" (as a shared community documentation project, first person pronouns sometimes make sense as a way of referring to Python developers or project contributors in general, but we don't want to use them to refer to specific community subsets)
• took a close look at anything containing other pronouns (third party pronouns are generally fine, but the text needs to be clear on the intended reference)

There are a few more substantial comments mixed in:

• suggesting a different discussion title
• one case where I think a comment in the text isn't accurate (for projects that support typechecking, downstreams probably should be running a static typecheck to check for potential problems with dependency upgrades)
• a couple of cases where I suggested added a new bullet point to the rationale for a section
• a few cases of more substantial suggested changes to the wording of various points, or adding entirely new paragraphs to the text

[mgorny]

Thanks for your feedback.

• suggested dropping most of the bold emphasis from the text (mostly dropped entirely, occasionally scaled back to just italics rather than bold)

Not sure about this. It's been pointed out to me that there are people who would like to skim the text rather than reading it in detail. Bold makes it easy to find the key points, especially if you've read it once and now just want to quickly consult.

• one case where I think a comment in the text isn't accurate (for projects that support typechecking, downstreams probably should be running a static typecheck to check for potential problems with dependency upgrades)

To be honest, type checking has never caused us anything but trouble. I suppose type checking may make sense if there is a poor test coverage. However, if the test coverage is reasonably good, any real breakage affecting users should be detected through regular tests. Type checking, on the other hand, will generally report things that are "technically wrong", but do not really affect affect end users (i.e. there is no hurry to backport fixes to stable versions of a package). On top of that, MyPy is being constantly improved, so packages that passed typing tests in the past, start failing with newer MyPy versions.

Hmm, perhaps I should also mentioned the existence of "stable" channels somewhere.

[mgorny]

• suggested rewording for anything containing the pronouns "you" or "your" (it's really hard to make second person pronouns sound non-confrontational in English, so it's simpler to avoid them entirely)

Perhaps https://packaging.python.org/en/latest/contribute/#conventions-and-mechanics needs to be updated for that, because using "you" is not my usual style, and I've done that specifically to follow the guidelines.

[mgorny]

Thanks for your review. I've applied most of the suggestions, left a few with comments.

[mgorny]

Okay, I've attempted to address the remaining suggestions (modulo the one about Rust), taking my comments into consideration. Hope that addresses your concerns. I'm going to add one more section now, regarding release stability.

[mgorny]

There seems to be some confusion around Internet access part too. I need to think how to clarify it to make it clear that I'm talking about the part of the build process performed by the backend, and not the way frontend works. Probably worth noting too that most of the time, people don't need to be concerned about that — it only applies if they explicitly do something like fetching vendored dependencies or running npm install ....

[pawamoy]

Other general suggestions:

• write "Git" instead of "git" everywhere
• use consistent casing in bullet points: sentence starts with capital letter, or not

Very nice document! Super interesting and helpful :)

[mgorny]

Hopefully this version is clearer. In particular, I've tried to emphasize that we're talking about explicitly fetching stuff such as vendored dependencies, and not installing dependencies as done by a build frontend.

[JelleZijlstra]

Thanks, this is very helpful!

[mgorny]

@JelleZijlstra, hope this addresses your concern. I've rewritten the paragraphs to make it clearer when I'm taking of PyPI, and when of some other hosting. I've also mentioned the possibility of installing tests.

[ncoghlan]

There are still a few inline conversations taking place (and the overall PR is still marked as a draft), so I'm not going to merge this yet, but I wanted to note that the proposed text is at a point now where I would be prepared to accept and publish it (we can always make further clarifications after its initial publication, after all).

[mgorny]

Thanks. I think I've addressed all the feedback so far to the best of my ability, either by applying changes or by commenting. I see GitHub hasn't marked some of the suggestions resolved, even though they were applied — unless I've missed one, they should be marked resolved now. So yeah, unless others have comments, I think this is ready to land.

[mgorny]

@ncoghlan, gentle ping. Can it be merged now?

[mgorny]

Gentle ping.

[mcepl]

Please, consider my comments for the further improvement of this text. Generally, I like it very much! Thank you!

[ncoghlan]

Thanks for the nudge, as checking back on the discussion here had indeed dropped off my radar.

Let's get this published, and then potentially iterate on specific subsections from there.

[mgorny]

Thank you!