docs: switch to the Jekyll JustTheDocs template

[damlobster]

Modify the documentation site to use the Jekyll template "Just The Docs". In my opinion this improves the usability of the site quite a bit and adds a search in the documentation (not perfect, but still useful). This modification is greatly inspired by the Fluidd documentation site.

I tried to keep the structure of the current site and to make as few changes as possible to the existing files, except for the following points:

• add the required front matter to all markdown files
• delete of line breaks in link texts because the template does not support it
• add index_{overview, configuration, developer, devices}.md files to build a hierachical navigation
• modify/add other Jekyll related files

The result is visible here: https://damlobster.github.io/klipper/

Signed-off-by: Damien Martin damlobster@gmail.com

[KevinOConnor]

Interesting. Thanks.

I agree the final web pages looks pretty good and the search function is very nice.

However, it looks like the automatic page table-of-contents on klipper3d.org (built from javascript) is no longer available (it was rendered to the left of the screen). That would be a notable loss for pages like Config_Reference.md .

add the required front matter to all markdown files

Unfortunately, this change is not something I'm comfortable with. Many people use the github rendering of the documentation (eg, https://github.com/damlobster/klipper/blob/docs_site/docs/API_Server.md ) and those links would render very strangely with this change. For a similar reason, I would not wish to add { toc } changes to the documentation.

I'm not sure if there is a way to get the new features without having to alter all of the individual .md files.

Thanks again,
-Kevin

[damlobster]

Thanks Kevin for your feedback.

However, it looks like the automatic page table-of-contents on klipper3d.org (built from javascript) is no longer available (it was rendered to the left of the screen). That would be a notable loss for pages like Config_Reference.md .

I totally forgot this one!
I added it back under the navigation (with a modification to highlight the currently visible headings). Not sure it's the best location and maybe someone with better web development skills could come up with a cleaner solution.

Unfortunately, this change is not something I'm comfortable with. Many people use the github rendering of the documentation (eg, https://github.com/damlobster/klipper/blob/docs_site/docs/API_Server.md ) and those links would render very strangely with this change.

Yes, I know it add a table at the beginning of each files. But as the goal is that the github repository is primilarly targeted to developers, I find it's acceptable for this audience.

I'm not sure if there is a way to get the new features without having to alter all of the individual .md files.

AFAIK, it's not possible only with Jekyll. The only way I see is to put all the metadata in one file and to setup a github workflow that reads this file, modify the .md files and push the result to a separate repository. www.klipper3d.org would be "served" from this new repository. But I think it's a little bit overkill...

For a similar reason, I would not wish to add { toc } changes to the documentation.

I reverted the back to the "manual" ToC.

[damlobster]

I found a solution to remove the jekyll metadata from the .md files and also put all the jekyll related files in docs/jekyll.

The code is in another branch: https://github.com/damlobster/klipper/tree/docs_site_with_workflow

The site is built, and the files pushed to a gh-pages branch, by this workflow: workflows/gh-pages-deploy.yaml.

Basically it first copy all the .md files and img + prints folders to the docs/jekyll folder. Then it uses the jekyll-action github action to build the site.

The metadata are defined in the file docs/jekyll/pages_frontmatter.yml. This file is merged with _config.yml during the build by Jekyll by specifying the two files as config files: jekyll_build_options: "--config _config.yml,pages_frontmatter.yml".

If you prefer this solution and want to update the documentation site, should I close the current PR and open a new one?

https://damlobster.github.io/klipper/ is updated with this version

[Sineos]

Looks quite cool 👍
On a high res screen, much space is wasted and there are strange line breaks:

[damlobster]

Thanks Sineos! It should now be better.

[KevinOConnor]

Thanks. As high-level feedback, the new web page looks nice to me.

Some comments:

1. I do prefer the second approach that does not require the "front matter" on the existing md documents.
2. If I understand this correctly, it would involve creating a gh-pages branch on github and a github "workflow" would automatically commit changes to that branch. I'm a little leery of that - it's not a stopper - but it is something I would go slow with.
3. If there are changes (like fixing links) that can be done first, it might be good to review and perform those changes first.
4. I did notice the autogenerated TOC contents look a little "out of place". I wonder if there is a better way to handle that.
5. Not sure if you've looked, but is it possible to extract out the search functionality without pulling in all the other "just the docs" changes?

Thanks again,
-Kevin

[damlobster]

Thanks Kevin.

2: yes it's the process. It's required as we have to insert the "front matter" before generating the html with jekyll. But it could also be seen as an advantage because we can see the log of the build process in the case of a problem.
3: I didn't check all the links, but except the ones that contained new lines chars (in .md files), all were working fine.
4: I agree, it's not the most natural location. I'm not sure what could be the best one.

• Putting it inside the navigation under the current page "folder" would not great for pages with a lot of content
• On large screens it could be placed on the right of the page content, but with the navigation on the left it's not possible on smaller screen
• We could also implement it as a unfoldable div that would be closed on small screens

Although the navigation is not essential and use quite a bit of space, I think that it's faster to navigate in the documentation with it than with Overview.md page.

5: I didn't look to how to extract search from the theme. But it would require for sure to deploy the site via github actions because a json index file of all the pages content has to be created (lunr.js is used in the browser to search in this index).

I am quite busy for the coming weeks, but I'll try to find better ways of placing the ToCs.
Damien

[damlobster]

@KevinOConnor,

I updated this PR with the changes done in the docs_site_with_workflow branch and added the following changes:

• page tocs are now integrated into the navigation. I think this is much better than the previous "out of place" location.
• the search bar remains at the top of the page when scrolling
• I forced the current version (v0.3.3) of JustTheDocs theme in _config.yml

As before, the publishing is done by the workflow gh-pages-deploy.yaml. But now, the script docs/jekyll/pages_insert_metadata.py is executed before the generation of the site to add the jekyll frontmatters to the markdown files. The metadata are in docs/jekyll/pages_metadata.ini. The previous solution that merged two jekyll configuration files caused problems with the search index file creation.

Regarding the extraction of the theme search functionality, I don't think it's worth it.
There is quite a bit of SASS code in the theme and filtering out the relevant parts would take time.
Also, the creation of the search index file is done in the Rakefile script and my ruby knowledge is quite limited.

It's deployed there: https://damlobster.github.io/klipper/

Let me know what you think about it.
Thanks

[KevinOConnor]

Thanks. My high-level feedback is that the new site looks very nice, and I agree it would be an improvement to convert the klipper3d.org site to it.

This is a big change to the documentation and I think it would be useful to make some minor changes before merging.

1. rebase and squash all the commits into one commit.
2. Don't change FAQ.md and Contact.md in this commit (nor Overview.md if possible).
3. I think the new files should pass the whitespace checks (no trailing spaces, no trailing newlines, no tabs, all files end with a newline).
4. I'm unsure about the directory name jekyll as I fear most people wont know what that is. If _layouts is too confusing, maybe _website or _klipper3d or something else?

Thanks for working on this.
-Kevin

[damlobster]

Great!
I made the changes you asked for and chose _klipper3d for the folder name.

The deployement workflow needs write access to the repository to be able to push the site to the gh-pages branch. To do this, you need to generate a token and add it as GH_TOKEN to the repository secrets. The procedure is explained here: https://jekyllrb.com/docs/continuous-integration/github-actions/#providing-permissions

You will also need to change the GitHub Pages source branch from master to gh-pages.

The deployement workflow is executed for all commits in the master branch that modify/add a file in the docs/ folder. I hope that everything will work without any problems!

[damlobster]

I missed a bug that was causing the table of contents of the page not to be displayed. It's fixed...

[damlobster]

@KevinOConnor, I just came across MkDocs and the very good Material theme: https://squidfunk.github.io/mkdocs-material/

It's unfortunate that I didn't saw it before because it supports out of the box everything that I modified (metadata outside markdown files, search + highlight, TOC, ...).

To test it locally:

1. install mkdocs + material theme: pip install mkdocs-material
2. create a file klipper/mkdocs.yml with this content

site_name: Klipper documentation
theme:
  name: material

3. launch mkdocs: mkdocs serve
4. then go to: http://127.0.0.1:8000

Using this solution would be much simpler, and surely more robust, than using Jekyll+JustTheDocs: it requires only a github workflow, a mkdocs.yml file and maybe a CSS file to customize the theme + some javascript to rewrite the links to ../config/.

Many md files will nevertheless need to be modified for search and TOC to work properly: each page must have a single H1 heading at the top containing the page title. But I think that these modifications would not impact negatively the rendering of the docs on github.

If you are interested, I can spend some time on this alternative solution.

[KevinOConnor]

Interesting. I don't have any love for Jekyll, so I don't see an issue with using a different rendering tool. If you want to investigate that, we can put the current implementation on hold for a couple of weeks to see what an alternative looks like.

Does mkdocs use the same "ghpages" branch trick for github pages? I can't seem to figure out what the ominous mkdocs gh-deploy --force command does from just their documentation.

Thanks.
-Kevin

[damlobster]

Ok, I think it's worth a try.

The name of the branch to which the documents are deployed can be specified in mkdocs.yml. I think it's also possible to push the generated site to another github repo.

I expect to be able to work on it by mid August. I will make a new PR as soon as I have a first working version.

[KevinOConnor]

I think this PR has been superseded by #4512.

-Kevin

[damlobster]

Of course, I close it!