Update Developer Guides (#3088)

(cherry picked from commit f7bd2ce)

Author: IgnatBeresnev

.github/workflows/gh-pages-deploy-mkdocs.yml .github/workflows/gh-pages-deploy-dev-docs.yml

@@ -1,11 +1,11 @@
-name: Deploy MkDocs to GitHub Pages
+name: Deploy developer docs to GitHub Pages
 
 on:
   push:
     branches:
       - master
     paths:
-      - 'mkdocs/**'
+      - 'docs-developer/**'
   release:
     types: [ published ]
 
@@ -25,7 +25,7 @@ jobs:
       - uses: gradle/gradle-build-action@v2
         with:
           gradle-home-cache-cleanup: true
-      - name: Get current dokka version
+      - name: Get current Dokka version
         run: echo "DOKKA_VERSION=`./gradlew :properties | grep '^version:.*' | cut -d ' ' -f 2`" >> $GITHUB_ENV
         working-directory: ./dokka
       - name: Build docs
@@ -35,6 +35,6 @@ jobs:
         uses: peaceiris/actions-gh-pages@v3
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
-          publish_dir: ./dokka/mkdocs/build/mkdocs
+          publish_dir: ./dokka/docs-developer/build/mkdocs
           keep_files: true
           full_commit_message: Publish ${{ env.DOKKA_VERSION }} documentation

docs-developer/README.md

@@ -0,0 +1,43 @@
+# Developer documentation
+
+This module contains developer documentation which is published to GitHub pages: 
+[kotlin.github.io/dokka](https://kotlin.github.io/dokka/).
+
+It is built using the [gradle-mkdocs-plugin](https://github.com/xvik/gradle-mkdocs-plugin).
+
+## Building
+
+You can build the documentation locally:
+
+```Bash
+./gradlew :docs-developer:mkdocsBuild
+```
+
+The output directory is `build/mkdocs`.
+
+### Docker
+
+Alternatively, you can use Docker:
+
+```bash
+docker run --rm -it -p 8000:8000 -v ./docs-developer/src/doc:/docs squidfunk/mkdocs-material
+```
+
+This will build the docs and start a web server under [localhost:8000/Kotlin/dokka](http://localhost:8000/Kotlin/dokka/).
+
+### Livereload server
+
+Alternatively, you can run a livereload server that automatically rebuilds documentation on every change:
+
+```Bash
+./gradlew :docs-developer:mkdocsServe
+```
+
+By default, it is run under [localhost:3001](http://localhost:3001/), but you can change it in 
+[mkdocs.yml](src/doc/mkdocs.yml) by setting the `dev_addr` option.
+
+## Publishing
+
+The documentation is published automatically for all changes in master and for every GitHub release.
+
+See [gh-pages.yml](../.github/workflows/gh-pages-deploy-dev-docs.yml) workflow configuration for more details.

mkdocs/build.gradle.kts docs-developer/build.gradle.kts

@@ -2,7 +2,7 @@ import org.jetbrains.dokkaVersionType
 import org.jetbrains.DokkaVersionType
 
 plugins {
-    id("ru.vyarus.mkdocs") version "2.3.0"
+    id("ru.vyarus.mkdocs") version "2.4.0"
 }
 
 if (dokkaVersionType != DokkaVersionType.RELEASE) {

docs-developer/src/doc/docs/developer_guide/architecture/architecture_overview.md

@@ -0,0 +1,123 @@
+# Architecture overview
+
+Normally, you would think that a tool like Dokka simply parses some programming language sources and generates
+HTML pages for whatever it sees along the way, with little to no abstractions. That would be the simplest and
+the most straightforward way to implement an API documentation engine.
+
+However, it was clear that Dokka may need to generate documentation from various sources (not only Kotlin), that users
+might request additional output formats (like Markdown), that users might need additional features like supporting
+custom KDoc tags or rendering [mermaid.js](https://mermaid.js.org/) diagrams - all these things would require changing 
+a lot of code inside Dokka itself if all solutions were hardcoded.
+
+For this reason, Dokka was built from the ground up to be easily extensible and customizable by adding several layers
+of abstractions to the data model, and by providing pluggable extension points, giving you the ability to introduce
+selective changes on a given level.
+
+## Overview of data model
+
+Generating API documentation begins with input source files (`.kt`, `.java`, etc) and ends with some output files
+(`.html`/`.md`, etc). However, to allow for extensibility and customization, several input and output independent
+abstractions have been added to the data model.
+
+Below you can find the general pipeline of processing data gathered from sources and the explanation for each stage.
+
+```mermaid
+flowchart TD
+    Input --> Documentables --> Pages --> Output
+```
+
+* `Input` - generalization of sources, by default Kotlin / Java sources, but could be virtually anything
+* [`Documentables`](data_model/documentable_model.md) - unified data model that represents _any_ parsed sources as a 
+  tree, independent of the source language. Examples of a `Documentable`: class, function, package, property, etc
+* [`Pages`](data_model/page_content.md) - universal model that represents output pages (e.g a function/property page) 
+  and the content it's composed of (lists, text, code blocks) that the users needs to see. Not to be confused with 
+  `.html` pages. Goes hand in hand with the so-called [Content model](data_model/page_content.md#content-model).
+* `Output` - specific output formats like HTML / Markdown / Javadoc and so on. This is a mapping of the pages/content 
+  model to a human-readable and visual representation. For instance:
+    * `PageNode` is mapped as 
+        * `.html` file for the HTML format
+        * `.md` file for the Markdown format
+    * `ContentList` is mapped as
+        * `<li>` / `<ul>` for the HTML format
+        * `1.` / `*` for the Markdown format
+    * `ContentCodeBlock` is mapped as
+        * `<code>` or `<pre>` with some CSS styles in the HTML format
+        * Text wrapped in triple backticks for the Markdown format
+
+    
+You, as a Dokka developer or a plugin writer, can use extension points to introduce selective changes to the
+model on one particular level without altering everything else. 
+
+For instance, if you wanted to make an annotation / function / class invisible in the final documentation, you would only
+need to modify the `Documentables` level by filtering undesirable declarations out. If you wanted to display all overloaded
+methods on the same page instead of on separate ones, you would only need to modify the `Pages` layer by merging multiple
+pages into one, and so on.
+
+For a deeper dive into Dokka's model with more examples and details,
+see sections about [Documentables](data_model/documentable_model.md) and [Page/Content](data_model/page_content.md)
+
+For an overview of existing extension points that let you transform Dokka's models, see 
+[Core extension points](extension_points/core_extension_points.md) and [Base extensions](extension_points/base_plugin.md).
+
+## Overview of extension points
+
+An _extension point_ usually represents a pluggable interface that performs an action during one of the stages of
+generating documentation. An _extension_ is, therefore, an implementation of the interface which is extending the
+extension point.
+
+You can create extension points, provide your own implementations (extensions) and configure them. All of
+this is possible with Dokka's plugin / extension point API.
+
+Here's a sneak peek of the DSL:
+
+```kotlin
+// declare your own plugin
+class MyPlugin : DokkaPlugin() {
+    // create an extension point for developers to use
+    val signatureProvider by extensionPoint<SignatureProvider>()
+
+    // provide a default implementation
+    val defaultSignatureProvider by extending {
+        signatureProvider with KotlinSignatureProvider()
+    }
+
+    // register our own extension in Dokka's Base plugin by overriding its default implementation
+    val dokkaBasePlugin by lazy { plugin<DokkaBase>() }
+    val multimoduleLocationProvider by extending {
+        (dokkaBasePlugin.locationProviderFactory
+                providing MultimoduleLocationProvider::Factory
+                override dokkaBasePlugin.locationProvider)
+    }
+}
+
+class MyExtension(val context: DokkaContext) {
+
+    // use an existing extension
+    val signatureProvider: SignatureProvider = context.plugin<MyPlugin>().querySingle { signatureProvider }
+
+    fun doSomething() {
+        signatureProvider.signature(..)
+    }
+}
+
+interface SignatureProvider {
+    fun signature(documentable: Documentable): List<ContentNode>
+}
+
+class KotlinSignatureProvider : SignatureProvider {
+    override fun signature(documentable: Documentable): List<ContentNode> = listOf()
+}
+```
+
+For a deeper dive into extensions and extension points, see [Introduction to Extensions](extension_points/extension_points.md).
+
+For an overview of existing extension points, see [Core extension points](extension_points/core_extension_points.md) and
+[Base extensions](extension_points/base_plugin.md).
+
+## Historical context
+
+This is a second iteration of Dokka that was built from scratch.
+
+If you want to learn more about why Dokka was redesigned this way, watch this great talk by Paweł Marks:
+[New Dokka - Designed for Fearless Creativity](https://www.youtube.com/watch?v=OvFoTRhqaKg). The general principles 
+and general architecture are the same, although it may be outdated in some areas, so please double-check.

mkdocs/src/doc/docs/developer_guide/architecture/data_model/documentables.md docs-developer/src/doc/docs/developer_guide/architecture/data_model/documentable_model.md

@@ -1,18 +1,22 @@
-# Documentables Model
+# Documentable Model
 
-Documentables represent data that is parsed from sources. Think of this data model as of something that could be
-seen or produced by a compiler frontend, it's not far off from the truth.
+The Documentable model represents the data that is parsed from some programming language sources. Think of this data as 
+of something that could be seen or produced by a compiler frontend, it's not far off from the truth.
 
-By default, documentables are parsed from `Descriptor` (for `Kotlin`)
-and [Psi](https://plugins.jetbrains.com/docs/intellij/psi.html)
-(for `Java`) models. Code-wise, you can have a look at following classes:
+By default, the documentables are created from:
 
-* `DefaultDescriptorToDocumentableTranslator` - responsible for `Kotlin` -> `Documentable` mapping
-* `DefaultPsiToDocumentableTranslator` - responsible for `Java` -> `Documentable` mapping
+* Descriptors (Kotlin's K1 compiler)
+* Symbols (Kotlin's K2 compiler)
+* [PSI](https://plugins.jetbrains.com/docs/intellij/psi.html) (Java's model). 
 
-Upon creation, it's a collection of trees, each with `DModule` as root.
+Code-wise, you can have a look at following classes:
 
-Take some arbitrary `Kotlin` source code that is located within the same module:
+* `DefaultDescriptorToDocumentableTranslator` - responsible for Kotlin -> `Documentable` mapping
+* `DefaultPsiToDocumentableTranslator` - responsible for Java -> `Documentable` mapping
+
+Upon creation, the documentable model represents a collection of trees, each with `DModule` as root.
+
+Take some arbitrary Kotlin source code that is located within the same module:
 
 ```kotlin
 // Package 1
@@ -28,7 +32,7 @@ enum class Enum { }
 val topLevelProperty: String
 ```
 
-This would be represented roughly as the following `Documentable` tree:
+This would be represented roughly as the following Documentable tree:
 
 ```mermaid
 flowchart TD
@@ -43,20 +47,23 @@ flowchart TD
     secondPackage --> secondPackageProperty[DProperty]
 ```
 
-At later stages of transformation, all trees are folded into one (by `DocumentableMerger`).
+At later stages of transformation, all trees are folded into one by 
+[DocumentableMerger](../extension_points/core_extension_points.md#documentablemerger).
 
 ## Documentable
 
-The main building block of documentables model is `Documentable` class. It's the base class for all more specific types
-that represent elements of parsed sources with mostly self-explanatory names (`DFunction`, `DPackage`, `DProperty`, etc)
-.
-`DClasslike` is the base class for class-like documentables such as `DClass`, `DEnum`, `DAnnotation`, etc.
+The main building block of the documentable model is the `Documentable` class. It is the base class for all more specific 
+types. All implementations represent elements of source code with mostly self-explanatory names: `DFunction`, 
+`DPackage`, `DProperty`, and so on.
+
+`DClasslike` is the base class for all class-like documentables, such as `DClass`, `DEnum`, `DAnnotation` and others.
+
+The contents of each documentable normally represent what you would see in the source code. 
 
-The contents of each documentable normally represent what you would see in source code. For instance, if you open
-`DClass`, you should find that it contains references to functions, properties, companion object, constructors and so
-on.
-`DEnum` should have references to enum entries, and `DPackage` can have references to both classlikes and top-level
-functions and properties (`Kotlin`-specific).
+For example, if you open
+`DClass`, you should find that it contains references to functions, properties, companion objects, constructors and so
+on. `DEnum` should have references to its entries, and `DPackage` can have references to both classlikes and top-level
+functions and properties (Kotlin-specific).
 
 Here's an example of a documentable:
 
@@ -85,7 +92,7 @@ data class DClass(
 
 ___
 
-There are three non-documentable classes that important for this model:
+There are three non-documentable classes that are important for this model:
 
 * `DRI`
 * `SourceSetDependent`
@@ -94,9 +101,9 @@ There are three non-documentable classes that important for this model:
 ### DRI
 
 `DRI` stans for _Dokka Resource Identifier_ - a unique value that identifies a specific `Documentable`.
-All references and relations between documentables (other than direct ownership) are described using `DRI`.
+All references and relations between the documentables (other than direct ownership) are described using `DRI`.
 
-For example, `DFunction` with a parameter of type `Foo` has only `Foo`'s `DRI`, not the actual reference
+For example, `DFunction` with a parameter of type `Foo` only has `Foo`'s `DRI`, but not the actual reference
 to `Foo`'s `Documentable` object.
 
 #### Example
@@ -146,11 +153,11 @@ kotlinx.coroutines/MainCoroutineDispatcher/limitedParallelism/#kotlin.Int/Pointi
 ### SourceSetDependent
 
 `SourceSetDependent` helps handling multiplatform data by associating platform-specific data (declared with either
-`expect` or `actual` modifier) with particular 
+`expect` or `actual` modifiers) with particular 
 [source sets](https://kotlinlang.org/docs/multiplatform-discover-project.html#source-sets).
 
-This comes in handy if `expect`/`actual` declarations differ. For instance, the default value for `actual` might differ
-from that declared in `expect`, or code comments written for `expect` might be different from what's written
+This comes in handy if the `expect` / `actual` declarations differ. For example, the default value for `actual` might 
+differ from that declared in `expect`, or code comments written for `expect` might be different from what's written
 for `actual`.
 
 Under the hood, it's a `typealias` to a `Map`:
@@ -171,18 +178,18 @@ ___
 
 ## Documentation model
 
-Documentation model is used alongside Documentables to store data obtained by parsing
-code comments (such as `KDoc`/`Javadoc`).
+The Documentation model is used alongside documentables to store data obtained by parsing
+code comments (such as KDocs / Javadocs).
 
 ### DocTag
 
 `DocTag` describes a specific documentation syntax element.
 
-It's universal across source languages. For instance, DocTag `B` is the same for `**bold**` in `Kotlin` and
-`<b>bold</b>` in `Java`.
+It's universal across language sources. For example, the DocTag `B` is the same for `**bold**` in Kotlin and
+`<b>bold</b>` in Java.
 
-However, some `DocTag` elements are specific to a certain language, there are many such examples for `Java`
-because it allows HTML tags inside `Javadoc` comments, some of which are simply not possible to reproduce with `Markdown`.
+However, some DocTag elements are specific to one language. There are many such examples for Java, because it allows 
+HTML tags inside the Javadoc comments, some of which are simply not possible to reproduce with Markdown that KDocs use.
 
 `DocTag` elements can be deeply nested with other `DocTag` children elements.
 
@@ -218,10 +225,9 @@ data class CodeBlock(
 
 ### TagWrapper
 
-`TagWrapper` describes the whole comment description or a specific comment tag.
-For example: `@see` / `@author` / `@return`.
+`TagWrapper` describes the whole comment description or a specific comment tag. For example: `@see` / `@author` / `@return`.
 
-Since each such section may contain formatted text inside of it, each `TagWrapper` has `DocTag` children.
+Since each such section may contain formatted text inside it, each `TagWrapper` has `DocTag` children.
 
 ```kotlin
 /**