Can't remove deleted pages from search or "current" URL

[aldeed]

🐛 Bug Report

Have you read the Contributing Guidelines on issues?

Yes, I highly recommend them

To Reproduce

1. Follow https://docusaurus.io/docs/en/installation to make a new docusaurus v1 project
2. cd website
3. yarn examples versions
4. yarn run version 1.0.0
5. Delete docs/doc3.md file and delete "Second Category": ["doc3"] from sidebars.json
6. yarn run version 2.0.0
7. yarn start
8. View the website in a browser, and see that /docs/doc3 still resolves to the version 1.0.0 file in a browser even though it doesn't appear in the sidebar.
9. If you can set up algolia, see that you can still get to the "current" URL that way, too.

Expected and Actual behavior

When I delete a markdown file from the main docs folder and remove it from the sidebar, I would expect it to be inaccessible from the "current" version.

But it continues to be returned by Algolia DocSearch, which can be clicked and the /docs/<id> does actually load correctly because of versioning fallback. This makes it appear as though that page still exists when it in fact does not.

Possible Solutions

Maybe the version command should be creating an empty markdown file in a versioned_docs folder for every file that is no longer in /docs?

Or maybe it can be solved manually with an array of doc IDs that should no longer be accessible?

Or if there isn't a clear solution to the problem of the page being accessible, it should at least not appear in Algolia results so that nothing links to it.

Your Environment

• Docusaurus version used: 1.14.4
• Environment name and version (e.g. Chrome 78.0.3904.108, Node.js 10.17.0):
  • Chrome 80.0.3987.132 (Official Build) (64-bit)
  • Node 8
• Operating system and version (desktop or mobile): MacOS Catalina, desktop

Reproducible Demo

https://github.com/aldeed/docusaurus-versioning-issue

This repo followed the reproduction steps above so you only need to clone it, yarn install, and then start with the yarn start step.

[yangshun]

Unfortunately this is due to an oversight in the design of v1. See #2123 for more info.

v2 uses a snapshotting approach which wouldn't have this problem.

[aldeed]

@yangshun I don't know how close v2 is to feature complete and how easy upgrading to it will be, but is there willingness to fix this in v1 using one of the approaches I suggest in "Possible Solutions" section above? For example, a list of deleted IDs (skipFallbackIDs?) in config seems simple and non-breaking. I could submit a PR but I'd need some help figuring out which areas of the code would need to check it.

[slorber]

@aldeed v2 is already used by a lot of people. Not so hard to upgrade.

versioning is not 100% stable but still works quite nicely, i18n is not ready.

if you submit a PR to v1 I could merge it. I'm not very familiar with v1 code to help you.

A workaround could be to delete the pages that you don't want in the ./build folder, it's just regular html files on a directory, so deleting them in a post-build step would probably enable the 404 page of your host to kick in:

[aldeed]

Thanks @slorber. There are no pages to delete. The problem is the fallback behavior so fixing it would involve deleting pages from previous versions, which is not what we want. We DO want the pages visible when viewing an old version (which has them in TOC) but we DO NOT want them visible when you search the current version or go directly to that ID URL in the current version. I'll see if I can figure out any easy way to circumvent the fallback from a list of IDs.

EDIT: Nevermind, I think I see what you are saying. Delete all the files/folders after the last version in which the content was correct. That would work, but it's quite a lot to delete. Need to delete from every subsequent version plus the current. If I can figure out how to not generate those in the first place, that will be a much quicker build.

[aldeed]

@yangshun @slorber I have a PR with a possible solution that works well for me. #2955