Add the ability to delete "legacy" documentation

[dstufft]

If a project has existing legacy documentation hosted on pythonhosted.org, we should allow them to delete this documentation. This should come with appropriate warnings that this is a one time, destructive operation that cannot be undone, that new uploads of documentation are no longer allowed and that they should not do it until they've moved their documentation hosting elsewhere.

We should probably include at least a suggestion of using ReadTheDocs here.

This should also provide people with the option to add a redirect when deleting their documentation so that they can redirect to their new location.

[nlhkabu]

@dstufft - is this required for shut down, or can we move it to post launch?

[dstufft]

It can be post shut down, it doesn't currently exist.

[dstufft]

Actually I lied-- We're not allowing new docs to be uploaded in Warehouse, so we'll want people to be able to delete their existing documentation so they can migrate to something else before we shut down legacy.

[bbayles]

Posting very late to an old thread, but adding my support for adding this ability. For more-itertools we added docs to RTD, but the old ones still show up on Google and there's a link to them on PyPI.

[bbayles]

Oh, there's a button for this! Never mind.

[computron]

This should also provide people with the option to add a redirect when deleting their documentation so that they can redirect to their new location.

Is there any redirect option yet? I've erased my pythonhosted docs (which can't be updated) and moved my docs elsewhere, but there are published academic papers and other materials that refer to documentation for my package as being on pythonhosted.org. Currently, following any of those links just returns an error page with no guidance of where to go next. I'm sure many users can eventually figure out where to go with Google, but it would be much better if pythonhosted allowed for a way to redirect or at least list the location of the updated docs.

[mperrin]

Let me second @computron's suggestion - it would be really nice to be able to redirect from the pythonhosted.org docs to our new docs now hosted on readthedocs.org. Thanks much.

[lingthio]

Wow. Seriously? This is such a major oversight. You expect me to choose between a bad option:

• Delete the pythonhosted.org page, so that my user base can't find it.

and a worse option:

• Live with two docs pages, one of them out-dated, which for a while will show up with higher search rankings?

Furthermore, google search offers almost no search results regarding this issue. It took me over an hour to find this. At least make this info available on the pypi edit project page. PLEASE.

[ncoghlan]

@dstufft, as per @lingthio's comment above, and the distutils-sig thread at https://mail.python.org/pipermail/distutils-sig/2017-September/031453.html, this is no longer a potential future problem, it's an actual problem right now: with pypi.org providing the only artifact upload interface, there's no longer any way for package maintainers to update their pythonhosted.org documentation.

Mere deletion isn't adequate, since that doesn't allow previously published links to be transparently redirected (preferable), or at least to reach a "These docs are no longer here, go to [other page] instead" page.

As prior art for handling this more gracefully, https://docs.readthedocs.io/en/latest/user-defined-redirects.html would be worth looking at.

It may even be worth chatting to @ericholscher to see whether or not it might be possible to migrate the pythonhosted.org pages into ReadTheDocs as static HTML, such that the redirect strategy literally is to use RTD's existing support for this.

[fungi]

As a compromise between adding a new complex interaction for user-defined redirects and/or parsing special links out of descriptions/metadata, would it be sufficient simply to redirect pythonhosted docs for any package missing them to the corresponding package page? Then at least if the project has a Documentation or Home Page link in their description or metadata the user can continue by following it to the current docs.

[ronaldoussoren]

Redirecting to an URL in the metadata would be nicer, either Home-page or a specific Project-URL (and the latter would give projects more control on where the documentation link redirects to).

In the longer run #1419, which is mostly about being able to add links to the sidebar on PyPI.org through Project-URL, is a nicer solution.

P.S. For a long time I didn't have links to documentation in my long-description because PyPI legacy automatically ads a link to documentation hosted on pythonhosted.org. I only recently added links to the long description because that doesn't work with warehouse and I couldn't get the same functionality through Project-URL links.

[ericholscher]

I'd be interested in helping get something set up.

[nlhkabu]

I've added a space for this in the new UI:

For now, this tab will be commented out, but this interface should give us the flexibility to add a specific documentation page and address this issue.

[di]

FYI, Project-URLs have been fixed in #2813.

[ewdurbin]

This is In Progress. Implementing the Delete Docs button is one part, but there is a larger story.

A discussion happened late last year with Read The Docs (RTD) about what a good approach for handling a link between documentation and the package index might be, as warehouse isn't well suited to serve as a documentation host moving forward.

The summary is that post launch, we may be able to implement some sort of handoff to "introduce" a user to RTD for the purposes of authenticating them as valid owners of a namespace on their site that indicates "canonical" package documentation hosting.

Implementing a "redirect" functionality is still up in the air, as it would effectively communicate that we're not going to do anything for package doc hosting moving forward... which hasn't been decided. Although I agree that it would be a good escape hatch in the interim before a more full fledged solution is implemented.

[brainwane]

Once we have this issue resolved, we should ping @jezdez and @WoLpH to update the sphinx-pypi-upload project & fork (relevant PR) appropriately.

[brainwane]

For context, since I think a few people subscribed to this issue don't know:

Warehouse, the codebase behind the new PyPI, is available as a pre-production site at https://pypi.org . Thanks to limited funding from Mozilla's Open Source Support Program, and it's on its way to replacing the legacy PyPI site by the end of April (roadmap). We're currently seeking feedback from package maintainers, about what does or doesn't work for you in the new interface, and in the next few weeks we'll be doing so with the wider Python developer community.

While the pythonhosted docs/Read the Docs "handoff" can’t really happen until we shut down legacy PyPI, we will be implementing the "delete existing docs" functionality once we (mostly Ernest) have completed the current foundational infrastructure work necessary for the Warehouse switchover. So you'll probably see this within the next six weeks.

I'm sorry for the delay.

[brainwane]

@dstufft said,

We're not allowing new docs to be uploaded in Warehouse, so we'll want people to be able to delete their existing documentation so they can migrate to something else before we shut down legacy.

Does shutting down pypi.python.org also involve shutting down https://www.pythonhosted.org/ and thus making pythonhosted.org documentation (example: https://pythonhosted.org/spyder/help.html ) completely unavailable to users? I presume it does not, in which case, this feature, though deeply needed, does not need to block the shutdown of legacy PyPI.

[ewdurbin]

once #3413 merges, we should refile a new issue for any redirect behavior that we might envision.

[pchtsp]

I'm sorry to highjack this thread but I'm going crazy trying to manually add a redirect on our old docs (https://pythonhosted.org/PuLP/) to our new documentation (http://coin-or.github.io/pulp/) following this stackoverflow's instructions but I can't for the life of me find any option in all of pypi to upload or delete the documentation. I'm maintainer in the PuLP project.

Am I supposed to be owner of the project to see this option? Has it been taken down? Is there anyway to update the current documentation in pypi to manually add a simple redirect? What are our current alternatives to redirect people from one site to the other?

Thanks!

Franco