Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

host Nix reference manual on nix.dev #764

Merged
merged 6 commits into from
Dec 4, 2023

Conversation

fricklerhandwerk
Copy link
Collaborator

this is the most cursed setup you will see any time soon.

we're dumping the Nix manual unchanged into the build tree after
building. the reason is that we'd want to link to it from our table of
contents, but because Sphinx does not allow adding arbitrary files to
the build output in arbitrary locations (only _static works). but we
want to place the manual behind a particular URL, and the alternative of
maintaining URL rewrites (e.g. in nginx) is not accessible here because the
infrastructure is managed somewhere else.

now that the files won't appear in their desired locations at Sphinx
build time, we can't use relative links to reference them, therefore we
have to resort to pointing to the web URL the manual will appear at.
this is terrible and I'm sorry. please fix this if you have a better
idea. once we use URLs though, we have to avoid linkchecking, since
those files may not be there before deploying them.

figuring all of this out took way longer than anyone would wish.

thanks @alejandrosame for enduring the process.

@fricklerhandwerk fricklerhandwerk requested a review from a team as a code owner October 18, 2023 13:59
@github-actions
Copy link
Contributor

Deploy Preview

Name Result
Last commit: 597acd72f57ed398bf4219c57a249a664d6afecd
Preview URL: https://d3c2049e.nix-dot-dev.pages.dev

@zmitchell
Copy link
Contributor

  • Why are we using the manual from 2.13 and not the current manual?
  • The sidebar link to the manual is broken

@fricklerhandwerk
Copy link
Collaborator Author

Why are we using the manual from 2.13 and not the current manual?

Because that's the version used in the latest NixOS release, while the most recent one will contain all kinds on things that don't exist there yet.

In the next steps we should figure out how to make other versions easily discoverable once they're available. One way would be to list them in the glossary where the manuals are also linked.

The link to the manual is broken only in the preview deployment, for reasons explained in the PR description.

@zmitchell
Copy link
Contributor

It definitely feels weird to me to have the Nix manual linked from nix.dev to be out of date compared to the one that you'll find by searching elsewhere. That's also likely to surprise users.

@fricklerhandwerk
Copy link
Collaborator Author

fricklerhandwerk commented Oct 20, 2023

Okay, so these are two different concerns. We can flip the version as we wish now that we have the infrastructure, that's cheap.

The more interesting question is how to present the fact that NixOS uses a particular version of Nix and that you'll likely want to use the manual for that rather than some random version from the future. One thing I considered was to actually pin the installation instructions to the same version as well. Because in the user journey we're building up, what may end up happening is that you install the latest version of Nix on Ubuntu, then add Home Manager which pins Nix to 2.13, and suddenly you can't run builds any more because the store DB schema is not forward compatible.

This is something I could already add in this PR to make the transition smooth for people starting out here. We can't do it in perfect lockstep, but the accompanying change will be to remove installation instructions and the manuals from nixos.org. The blocker here is though that I'd first want to prepare the Nixpkgs and NixOS manual upstream builds to use the right styling, as otherwise we'd have to process them here again. This is something I'm also currently working on, but it will take a bit of time, maybe another week or two on the wall-clock depending on distractions, as I have multiple ongoing commitments.

@zmitchell would you suggest to block the incremental change on everything else being finished, to avoid inconsistency? I'd prefer a temporary inconsistency to get it off my stack.

@zmitchell
Copy link
Contributor

So you could do all of that, or you could just have two links:

  • Nix Manual (Current)
  • Nix Manual (NixOS 23.05)

That seems like a better solution

@fricklerhandwerk
Copy link
Collaborator Author

fricklerhandwerk commented Oct 27, 2023

Okay, added a separate page where we can tall all we want about the Nix manual. @yukiisbored is there a way to force-expand only one entry of the TOC in the sidebar? Then users wouldn't have to click through to the page.

But I think this is useful now. @thufschmitt please take a look.

@github-actions
Copy link
Contributor

Deploy Preview

Name Result
Last commit: 3fea7ff960d8f229c726313de6c570a12cb040d5
Preview URL: https://6f63204d.nix-dot-dev.pages.dev

Copy link
Member

@thufschmitt thufschmitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @fricklerhandwerk !

What's the plan from the nixos.org side? Will we keep the manual on both websites or do we have a path towards redirecting the manual from nixos.org to here?

flake.nix Outdated Show resolved Hide resolved
@fricklerhandwerk
Copy link
Collaborator Author

fricklerhandwerk commented Oct 27, 2023

Once this is merged I'll migrate the per-page redirects here, and then put in place a manual-wide redirect on nixos.org.

Eventually the page redirects should move upstream into the Nix manual, but that needs a bit of figuring things out.

@NixOS NixOS deleted a comment from github-actions bot Oct 28, 2023
@asymmetric
Copy link
Contributor

@fricklerhandwerk is it known that the links to the manual for 2.13 and 2.18 are broken?

@fricklerhandwerk
Copy link
Collaborator Author

fricklerhandwerk commented Oct 29, 2023

@fricklerhandwerk is it known that the links to the manual for 2.13 and 2.18 are broken?

The public URLs cannot be populated before deploying this PR. See the commit message for why it's done this way. The deploy preview hosts them under the relative path: https://6f63204d.nix-dot-dev.pages.dev/manual/nix/2.13/

@yukiisbored
Copy link
Member

@yukiisbored is there a way to force-expand only one entry of the TOC in the sidebar?

I don't think that is possible to do with Sphinx TOC.

Though, mdbook has the version widget maybe we should look into using that instead.

image

@fricklerhandwerk
Copy link
Collaborator Author

That version widget is slapped on top by nixos.org. I'd prefer to avoid porting that and do more things statically. That's easier to find, to understand and to maintain. (Documentation should be low-tech!)

@infinisil
Copy link
Member

This feels too hacky to me :/

@fricklerhandwerk
Copy link
Collaborator Author

fricklerhandwerk commented Nov 2, 2023

@infinisil @zmitchell yes, this is hacky and I'm open to suggestions. Opened NixOS/nix#9275 as a pathway to fix forward.

@thufschmitt added another Nix release.

Copy link
Contributor

github-actions bot commented Nov 2, 2023

Deploy Preview

Name Result
Last commit: 6c18a10211980711818725a27277bc04dfb9822c
Preview URL: https://2dae7df6.nix-dot-dev.pages.dev

@NixOS NixOS deleted a comment from github-actions bot Nov 2, 2023
Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should document everything that needs to be done regularly:

  • Updating Flake inputs
  • Adding new Flake inputs
  • Doing string replacement on the URLs

flake.nix Outdated Show resolved Hide resolved
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-11-02-documentation-team-meeting-notes-91/34938/1

Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a rebase, but otherwise this looks okay to me.

fricklerhandwerk and others added 5 commits November 27, 2023 07:30
this is the most cursed setup you will see any time soon.

we're dumping the Nix manual unchanged into the build tree *after*
building. the reason is that we'd want to link to it from our table of
contents, but because Sphinx does not allow adding arbitrary files to
the build output in arbitrary locations (only `_static` works). but we
want to place the manual behind a particular URL, and the alternative of
maintaining URL rewrites (e.g. in nginx) is not accessible here because the
infrastructure is managed somewhere else.

now that the files won't appear in their desired locations at Sphinx
build time, we can't use relative links to reference them, therefore we
have to resort to pointing to the web URL the manual will appear at.
this is terrible and I'm sorry. please fix this if you have a better
idea. once we use URLs though, we have to avoid linkchecking, since
those files may not be there before deploying them.

figuring all of this out took way longer than anyone would wish.

Co-authored-by: Alejandro Sanchez Medina <alejandrosanchzmedina@gmail.com>
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-11-27-nix-team-meeting-minutes-107/36112/1

Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts:

  • Add redirects for /manual/nix/stable and the nixos releases? We'd first have to decide what the names should even be, so let's postpone.
  • Maybe get the manuals from Nixpkgs if applicable? Not sure, because we can't do that for all versions we want to show, and we'd have to pick flake vs Nixpkgs arbitrarily for most versions.
  • Commit the generated sites, so that the build process and particularly the evaluation, are more scalable? Not essential and has its own complexity just like the current PR approach. Let's try this PR first.

100% love the concept. nix.dev isn't nixos.org and Nix isn't NixOS, so that gets rid of what was arguably a misnomer. Also explicit versions are good. We might want to keep more versions around, but that's not a matter of urgency, as mentioned.

Code looks ok. Let's see how this goes in practice.

@roberth roberth merged commit 9cc2ddd into NixOS:master Dec 4, 2023
5 checks passed
@edolstra
Copy link
Member

edolstra commented Dec 5, 2023

This is really confusing because now we have two canonical locations for the Nix manual, since https://nixos.org/learn points at the Nix manual at https://nixos.org/manual/nix/stable/.

nix.dev isn't nixos.org and Nix isn't NixOS

nixos.org is the official homepage for Nix. Changing that should be a decision for the marketing team.

Also, this PR makes me have little faith in the ability of nix.dev to represent the direction of the Nix project.

@fricklerhandwerk
Copy link
Collaborator Author

fricklerhandwerk commented Dec 6, 2023

now we have two canonical locations for the Nix manual, since https://nixos.org/learn points at the Nix manual at https://nixos.org/manual/nix/stable/.

This is very much temporary, and can't be switched over atomically anyway, because the moving parts live in different repositories. It will be fixed by adding a redirect from nixos.org after merging #829.

@06kellyjac
Copy link
Member

On the marketing team comment I can agree but for all I know there might have been discussions with the marketing team already.


Also, this PR ...

I like using flakes and I personally would have just kept using them, but it is objectively true that there are users that don't use experimental features and or aren't happy with flakes yet.

The nix project "represents" flakes as "experimental" so it is completely fair for a project representing nix to show the other options currently in use.

Even if the experimental flag was removed are you suggesting every single project under the nixos org umbrella must use them?

The impression I've got from the recent calls to remove the experimental tag is "well flakes are in a good spot, we can continue improving them when non-experimental, you can use flakes or not just like you can now and its totally a non issue" but this kind of goes against that important last part.

It only makes sense to enforce the usage of flakes for the nixos org projects if the non-flakes approach are explicitly viewed as legacy or deprecated.

@asymmetric
Copy link
Contributor

the reason is that we'd want to link to it from our table of
contents, but because Sphinx does not allow adding arbitrary files to
the build output in arbitrary locations (only _static works). but we
want to place the manual behind a particular URL, and the alternative of
maintaining URL rewrites (e.g. in nginx) is not accessible here because the
infrastructure is managed somewhere else.

@fricklerhandwerk there is a way actually:

  • placing the rendered 2.19 manual in e.g. ./source/_static/manuals/2.19
  • linking to it with bare <a> elements (because sphinx/myst don't know about it, so stuff like this doesn't work
  • adding a bunch of symlinks like ./source/_static/manuals/latest -> 2.19

This works (at least locally), with a few downsides:

  • the rendered maual is big (32MiB, which can be reduced to ~22 by removing a few files)
  • because of this, the git repo size would go up
  • we need to set up a workflow that does the manual size reduction
  • the sidebar links to the manuals would need to remain hardcoded to https://nix.dev (due to limitations with the toctree directive)

Advantages:

  • we can use relative links
  • less Nix magic to build the wbsite???

@fricklerhandwerk
Copy link
Collaborator Author

That sounds like an even worse hack. 😐

@fricklerhandwerk fricklerhandwerk deleted the add-manuals branch February 6, 2024 06:53
tomodachi94 added a commit to tomodachi94/nixpkgs that referenced this pull request Aug 17, 2024
The move happened here: NixOS/nix.dev#764

This is necessary because some redirects broke during the move.

Command used to get this result:
```
grep -rlF 'https://nixos.org/manual/nix/' ./ | xargs -I@ sed -i -e 's/https:\/\/nixos.org\/manual\/nix\/stable/https:\/\/nix.dev\/manual\/nix\/latest/g' @
```

For reviewers, here is a good way to ensure the updated links function:
```
nix-shell -p lychee --run "lychee doc/ --exclude '.*' --include 'https://nix.dev'"
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants