doc: create centralized documentation for developers

[jose-pepe-bsas]

Developers can easily access documentation on Puter project development concerns, and learn from it in order to contribute to ensure greater usefulness of their work.

For the use of it, there are two options: Either the developer runs the mkdocs server locally, or the puter repository can provide a public deployment to access. If you decide that the second case is useful please let me know and I could create a github actions pipeline to deploy this in a separate branch as a static site. Thx for read.

This documentation is in plain text markdown format to be portable in case you decide to change the documentation automation tool.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

jose seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[KernelDeimos]

Hi, the initiative in improving developer documentation is appreciated because that's something we would like to improve. As it stands right now, this PR needs several changes before we can consider merging it.

First, there's a principle in software development called Single Source of Truth; while this usually applies to code, I think it should apply to documentation as well. You've copied a number of files, leaving identical copies that could potentially go out of sync. Someone might read my documentation of the boot sequence from src/backend/doc/contributors/boot-sequence.md after somebody updates your copy at doc/dev_doc/docs/deployment/backend_boot_sequence.md. In your attempt to tidy things up, you might actually create more of a mess by doing this.

In the video you posted on our community Discord server, you've shown the merit of consolidating documentation into its own directory tree because we can use a documentation site builder. We do have a website at https://docs.puter.com and I think maybe we should instead expand the scope of this page. I'll talk with the others about making the repository for this accessible.

Your PR also raises an interesting question that needs some discussion: do we want docs under their respective modules (like how I have src/backend/doc for backend docs), interwoven in the source tree (like Godoc), or in a parallel directory structure (what you did in this PR)?

Here's what I think: We delete the contents of our Github Wiki (moving anything that's not in the Puter repo itself to the Puter repo, so without actually removing documentation) and make a script that populates it with documentation that takes the second approach I mentioned: docs are interwoven in the source tree. Our single-source-of-truth is then docs in the source tree, but tools like the one you showed us can be used by cloning the repo GitHub makes for the wiki page (which is already a submodule of the Puter repo itself). The advantage here is high cohesion between code and the relevant docs, which means when someone changes that code they're more likely to update the docs.

[jose-pepe-bsas]

I think the proposal you make would make it easier to keep a single source of truth about the project up to date (because keeps things easier while coding and low-level design), and does it programmatically.
The problem will be how to make it maintainable and coherent, documentation is often a facility for onboarding but can be tedious for the team.

Why I didn't try to collaborate in https://docs.puter.com instead? besides not having access, it was mainly because when I found it about three days ago, I didn't read anything about basic architecture issues in which to support me to collaborate in the project without overloading the Discord of the project with questions. The project has a large number of contributors and from what I read they have maintained a good dynamic, so I understand that they are interested in improving the onboarding of contributors 👍

To do so, I think it is advisable that we separate the high level documentation from the code implementation related documentation 📝

With separating it, I mean how and when it is done 🖊️ , allowing to specify what is being implementing in a code source element (at least its public interface, what you have already been doing) and interrelate it with a higher level documentation, with specific definitions of architecture (such as used patterns, or general project's config) but the most decoupling and abstract as possible to implementations.

💻 While programming a module or modifying one, the documentation, README.md files and in-code specs, have generated today is useful; it makes documenting easier, and allows executable documentation (for example if swagger, apidoc, or other tools were to be incorporated) and also keeps it updated when code is updated.

However, 🏗️ high-level concerns should not be documented on the fly while programming, because while programming one's attention is on a module and its tests, and not on its implications for the system.

💡 What I'm suggesting: for maintain the relationship between the high level and its implementations, I suggest that

1. we complete the architectural documentation (incorporating important and missing things like considerations and practices for testing, or how the project configuration works, among other generalities)
2. and then we can dynamically generate links between this documentation and the specifications that would be distributed in the source tree. A spec could be a README.md file corresponding to X module, or new jsdoc commentaries in our source code files.

We can in many ways to make that those Commits what modify the documentation to be lifted and sent to a Builder website (as used by MKDOCS) for example by shooting actions in a GH-Actions pipeline based on the name of Commit or modified files.

[KernelDeimos]

high-level concerns should not be documented on the fly while programming, because while programming one's attention is on a module and its tests, and not on its implications for the system.

I agree with this, but this does not mean documentation of high-level concerns doesn't belong in the source tree. The source tree is hierarchical, so higher-level documentation would go in the deepest directory that satisfies the criterion that everything the documentation covers is in the same directory. For example, docs describing the highest-level information about Puter's backend could go in /src/backend/doc while documentation describing the highest-level information about Puter could go in /doc. Documentation about Puter's AI module goes in /src/backend/module/puterai/doc

[KernelDeimos]

update: our GitHub wiki is now generated from doc directories in the repository. To consolidate changes in this PR, here's what needs to be changed:

• remove duplicated docs
• move new docs to follow the current convention (either in doc/contributors instead of doc/dev_doc/docs)
• update mkdocs.yml accordingly. If mkdocs expects documents to be in a flat directory structure, that's fine: point it to submodules/wiki.

If you make these changes we can merge this PR.

[jose-pepe-bsas]

I removed duplicity and moved the documents according to what I understood you are looking for. To facilitate changes in case you want to reorder some of what I have modified, I am sharing the small commits.

About mkdocs, I don't understand what is the point of using submodules/wiki and I haven't found any comments or discussion about it, could you explain it to me?