Proposal before PR: re-arrange top-level structure of docs

[jacalata]

I'd like to make some updates to the content structure of the docs. Before doing the work, I want to check if this sounds good to existing contributors, or if there are other plans that this might conflict with (as suggested in this issue)

starting with all these top-level pages:

• delete the page Building and Distributing...
  -- merge Feature Highlights and Forum+Bug Tracker sections into the index page.
  -- TBD the Transitional Note/setup.cfg/configuration API sections (all outdated?), could delete or merge into other detail pages.
• move Build System Support page into the User Guide section as "Dynamic Build Dependencies".
  --Remove "what is it" and "how to use it" paragraphs and delete or merge them into Quickstart/the index page.
• move pkg_resources page to the Deprecated section
• move keywords page to the Deprecated section

This would leave a top-level layout (under index.html) like this:

• User Guide
  --[Quickstart, Dependencies, Data files .....]
• Roadmap (<-- could be replaced with a link to github on the index page?)
• Development Guide
  --[Developer's Guide, Release Process]
• Backward Compatibility/Deprecated
  --[10+ child pages]
• History (<-- would like to rename this. Release notes? changelog?)

To reduce link breakage, ideally each page can be replaced with a 301 redirect, either to its new location or the closest remaining equivalent.

If this all goes well, I would try some similar re-arranging/condensing of content within the User/Development/Deprecated sections.

[jacalata]

The increased content on index.html combined with reducing the available start points for a reader should help address this issue: #3105

[codeandfire]

@codeandfire, wouldn't that produce a document that is too big and too scary?

@abravalheri basically, I'm intending this page to serve as some sort of "quick keyword reference" or a "cheatsheet" and a concise summary of all configuration/metadata options that Setuptools provides. To give you an idea, what I have in mind is roughly the following blueprint:

---

(START-OF-PAGE)

Build-Backend Specification
(pyproject.toml snippet to set setuptools.build_meta as the build backend)

Project Metadata

In setup.cfg, all project metadata must be listed under the [metadata] section.
In pyproject.toml, all project metadata must be listed under the [project] table.

Description	Keyword	Data type	Details
Name of the package	name	string	This name must be the name you expect your package to have on PyPI.
Version of the package	version	string	Can also be set dynamically. Refer ... for more details.
Keywords for PyPI	keywords	list of strings	...
...

Dependencies

Description	Keyword in setup.py	Keyword in setup.cfg	Keyword in pyproject.toml	Data type
Python dependencies for your package	install_requires	install_requires under the [options] section	dependencies under the project table	list of strings
Optional Python dependencies for your package	...	...	...	...
...

Package Discovery
(Similar to Dependencies section)

Data Files
(Similar to Dependencies section)

Entry Points
(Similar to Dependencies section)

Extension Modules
(Brief summary)

(Any other details to finish with)

(END-OF-PAGE)

---

This is a very rough idea, we can refine it further ...

We can avoid including too many details on any of these keywords here, and refer the reader to the appropriate User Guide page in case they would like to learn more about it. As far as Project Metadata is concerned, possibly we can create a separate User Guide page on this topic where we can discuss all of the corresponding options (name, version etc.) at length.
I also observed that a lot of material in the setup.cfg page and the pyproject.toml page overlaps with material given elsewhere in the User Guide: possibly we can restructure this material, removing anything that is redundant, and moving other details to the appropriate User Guide page.

I think in this way - while the page may be a little lengthy - it will not be overwhelming, especially for more experienced users who just want to look up a particular option.

[codeandfire]

I agree that this page is not in the right position. I would think that the "build_meta tweaks" section belongs to sort of "Advanced Usage" for heavy technical content. I am struggling to think about how to organize it though.
I had a look on the https://diataxis.fr/ framework, and I like its ideas in general (specifically the fact that we should optimise different pages with different types of readers in mind, as summarized greatly in this discussion).
However I don't want to have to do a complete overhaul of the docs (and I would like to preserve the URLs as much as possible).
Do you guys have any suggestion?

@abravalheri in the case of an Advanced Usage page, I would suggest including the material on Extending or Customizing Setuptools in that page, along with the material on build_meta "tweaks", because according to my understanding the material on the former page will typically be relevant only for plugin authors, and is therefore "advanced" in the sense that "consumers" will not require it.

[abravalheri]

Thanks @codeandfire, I really like the suggestion of having a cheatsheet, and it addresses one of the concerns that the diatáxis framework exposes: some people just want a quick reference to get things done.

With that in place we should be safely able to move the existing keywords page to the "deprecated guides".

I wonder though if the best approach is to have different columns in the same table or simply separate them with tabs. (Tabs are bad because they require duplication of the content, but several columns can be confusing for starters --- maybe we need some custom javascript to dynamically hide/show tables?).

I also agree that the Extending or Customizing Setuptools should also go to the "Advanced Usage" page.

All these ideas are great guys. I am highly supportive of it!

If I were to dream about an organisation that we could have at the docs, I think the following would be nice:

.
├── Quickstart  # Maybe we should bubble it up to be very visible?
├── Tutorials
│   ├── Distributing your first Python library
│   ├── Creating your first C-extension
│   └── Upgrading from older versions of Distutils/Setuptools
├── Configuration cheatsheet  # Maybe it should be the first item in the user guide? But I like it at the top-level...
├── User guide  # <-- most of the it looks like it could be classified as "how to" in diatáxis methodology
│   ├── Python packaging at a glance  # <-- This would be an introductory "Explanation" document
│   ├── Project organization and package discovery
│   ├── Controlling files in the distribution
│   ├── Dependency management
│   ├── Entry Points
│   ├── Data files support
│   ├── Specifying your project version
│   ├── Building extension modules
│   ├── Development mode (a.k.a editable installs)
│   └── Patterns and Anti-patterns
└── Advanced Topics
    ├── Making your own plugin
    ├── Dynamic build dependencies and other tweaks
    ├── Release cycle and API Reference
    ├── Setuptools architecture and internal Design
    ├── Bootstrapping and de-vendoring.
    └── Guides on backward compatibility & deprecated practice
        ├── ...
        └── Package Discovery and Resource Access using pkg_resources

Project
├── How to contribute?  # <-- I would love this one to be at the top level, I am not sure about the others.
├── Releases
├── Credit
└── Artwork

I hope one day we can get there...

[codeandfire]

@abravalheri the docs organization that you have suggested is quite good. I think @jacalata's proposal of reorganizing and restructuring pages is a good start.

Regarding tables/tabs: sure, I'm open to using tabs as well. The tables idea was quite rough - we do need think more about better ways of presenting the cheatsheet.

[abravalheri]

Thanks @codeandfire, just to clarify the tables idea is very good and nice visually. I think that even if we go for tabs we should use tables. The idea of using tabs is just to hide the several different keywords from setup.py, setup.cfg and pyproject.toml that can be confusing.

[abravalheri]

BTW, anyone wanting to work on this topic may try to apply for funding... I don't know how it works, but improving setuptools docs is currently listed as a fundable activity in the PSF: https://github.com/psf/fundable-packaging-improvements/blob/master/FUNDABLES.md#de-duplicate-and-reorganize-setuptools-documentation

[jacalata]

ok, part 1: remove Building and Distributing page #3405