Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor project structure and introduce composite builds #3174

Merged
merged 81 commits into from
Nov 10, 2023

Conversation

IgnatBeresnev
Copy link
Member

@IgnatBeresnev IgnatBeresnev commented Sep 5, 2023

At the moment, the project's build configuration is very coupled, which becomes problematic if one of the subprojects (like the Gradle plugin) needs to be more independent and be configured differently than the rest of the subprojects.

This PR serves as preparation work for #3131 (it will help with merging Dokkatoo into Dokka), and at the same time addresses several other issues:

  • Make use of Gradle's Composite builds, which are configured and executed in isolation.
  • Refactor the project structure:
    • Re-group the subprojects based on responsibility / layer [(analysis, core, plugin, runner).
    • Flatten subprojects (i.e core/test-api should become core-test-api)
  • Some refactoring of the build scripts to be more idiomatic, which will help with adopting modern Gradle features

Not in scope:


This PR will be in the Draft state until the build is green and the tests pass. Should try to keep the commits more isolated so that they can be reviewed separately.

aSemy and others added 14 commits September 5, 2023 16:02
* Move versioning plugin
* Move templating plugin
* Move mathjax plugin
* Move kotlin-as-java plugin
* Move all-modules-plugin
* Move android-documentation plugin
* Move base-test-utils
* Move base-frontend
* Move base plugin
* Move gfm-template-processing plugin
* Move the gfm plugin
* Move jekyll-template-processing plugin
* Move the jekyll plugin
@IgnatBeresnev IgnatBeresnev added the infrastructure Everything related to builds tools, CI configurations and project tooling label Sep 5, 2023
@IgnatBeresnev
Copy link
Member Author

I've relocated all of the currently existing subprojects to adhere to the desired structure, so now we just gotta make it work 😅

aSemy and others added 7 commits September 6, 2023 13:16
* add version-catalog def in integration-tests
* move integration-test shared utils into a subproject
* add root build.gradle.kts (even if it is empty)
* add `gradle.properties` to included builds
…efactoring

# Conflicts:
#	dokka-runners/gradle-plugin-classic/src/main/kotlin/org/jetbrains/dokka/gradle/kotlin/KotlinNativeDistributionAccessor.kt
@IgnatBeresnev IgnatBeresnev added this to the Dokka 1.9.20 milestone Oct 19, 2023
@IgnatBeresnev IgnatBeresnev mentioned this pull request Oct 19, 2023
@IgnatBeresnev IgnatBeresnev marked this pull request as ready for review November 9, 2023 12:17
The name of the root project was incorrect due to copy-paste, so it seems like either IDEA or git applied the patch from master to the wrong Gradle project, which caused integration test failures
@IgnatBeresnev
Copy link
Member Author

The PR is ready for review and ready to be merged.

Unfortunately, because everything is so tightly coupled and outdated, the scope grew and this PR also includes:

How to review it

Diff

It may seem like a lot of files have been changed, but the overwhelming majority of them were simply moved with git mv, which preserves history. If you exclude the commits that move files and resolve merge conflicts, you will end up with a reviewable list of changes.

I've excluded meaningless commits and generated a .patch file, and thankfully GitHub's gists know how to render them in a nice way, so you can have a look at the following gist for meaningful changes in this PR: https://gist.github.com/IgnatBeresnev/3ed43625bc01ea22421290458b9b2755

Another way to review it is to open it in IDEA and only select the following commits:

List of meaningful commits
  • Update hardcoded project paths
  • Update coroutine's diff
  • Fix Dokka version in coroutines and serialization tests
  • Fix integration tests
  • Fix publishing after the update to Gradle 8
  • Add parent validatePlugins task
  • Fix CLI integration tests
  • Make dokka-subproject a non-included build
  • Return a lost @Optional annotation
  • Make integration tests runnable from IDEA
  • Enable parent publishPlugins task
  • Remove gradle-plugin subproject to minimize diff
  • Minor build scripts refactoring
  • Minor convention plugin refactoring
  • Rename convention plugins
  • Remove check's dependency on integration tests
  • Replace deprecated API
  • fix s3 script path
  • run clean task separately
  • ignore some tests for now
  • fix gradle plugin compatibility
  • restore BCV
  • fix workflows
  • update gradle to 8.4 (not fully yet)
  • add parent tasks to dokka-integration-tests
  • cli integration tests
  • fix maven plugin POM
  • maven integration tests
  • rollback gradle integration tests to use mavenLocal for now
  • restore gradle integration tests
  • remove todos
  • fix publishing of Gradle plugin and it markers
  • cleanup build scripts setup group and version in properties setup publishing for gradle plugin
  • Correct publishing information
  • Add parent composite build tasks
  • Correct publishing env properties
  • Setup publishing to multiple repositories (gradle-plugin publishing is not supported yet)
  • Rip out publishing
  • Rip out non essentials
  • Fix dokka-core artifact name for runners
  • Move gradle-plugin's versions into the common catalog
  • Rip out non-essential dependencies in gradle-plugin
  • Align gradle-plugin's formatting with the rest of the project
  • Move parts of Dokkatoo into gradle-plugin
  • Align included build toolchain and language versions
  • Add composite build parent tasks
  • Fix Gradle sync problems
  • Update dokka_version to 1.9.20-SNAPSHOT
  • Remove irrelevant project properties
  • Add Gradle properties to included builds
  • Add version-catalog def in integration-tests
  • Initial included-build refactor

Places in code

If you'd prefer not to look at the diff and look at the code instead, the only really meaningful directories and files that need to be reviewed:

  • The overall project structure (where the modules are and how they are named)
  • All convention plugins inside ~/IdeaProjects/dokka/build-logic/src/main/kotlin
  • ~/IdeaProjects/dokka/build.gradle.kts

The rest of the changes were necessary to make the project buildable and pass integration tests.

…efactoring

# Conflicts:
#	dokka-integration-tests/gradle/src/integrationTest/kotlin/org/jetbrains/dokka/it/gradle/WasmJsWasiGradleIntegrationTest.kt
Copy link
Collaborator

@whyoleg whyoleg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's finally merge it :)

@IgnatBeresnev IgnatBeresnev merged commit 8e5c63d into master Nov 10, 2023
11 checks passed
@IgnatBeresnev IgnatBeresnev deleted the project-structure-refactoring branch November 10, 2023 10:46
IgnatBeresnev added a commit that referenced this pull request Nov 10, 2023
github-merge-queue bot referenced this pull request in slackhq/foundry Mar 5, 2024
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [org.jetbrains.dokka](https://togithub.com/Kotlin/dokka) | plugin |
patch | `1.9.10` -> `1.9.20` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>Kotlin/dokka (org.jetbrains.dokka)</summary>

### [`v1.9.20`](https://togithub.com/Kotlin/dokka/releases/tag/v1.9.20):
1.9.20

#### General bugfixes

- Fixed sealed interfaces not having the `sealed` keyword in signatures
([https://github.com/Kotlin/dokka/issues/2994](https://togithub.com/Kotlin/dokka/issues/2994))
- Fixed incorrect links in multi-module projects with non-unique package
names
([https://github.com/Kotlin/dokka/issues/2272](https://togithub.com/Kotlin/dokka/issues/2272)).
Huge thanks to [@&#8203;EddieRingle](https://togithub.com/EddieRingle)!
- Fixed member extensions not being shown on index pages in certain
scenarios
([https://github.com/Kotlin/dokka/issues/3187](https://togithub.com/Kotlin/dokka/issues/3187))
- Fixed Java's inner classes not having the `inner` keyword in Kotlin
signatures
([https://github.com/Kotlin/dokka/issues/2793](https://togithub.com/Kotlin/dokka/issues/2793))
- Fixed Java's `@param` tag not working with type parameters
([https://github.com/Kotlin/dokka/issues/3199](https://togithub.com/Kotlin/dokka/issues/3199))
- Fixed Dokka failing in KMP projects when the JVM source set is
suppressed
([https://github.com/Kotlin/dokka/issues/3209](https://togithub.com/Kotlin/dokka/issues/3209))

#### HTML format

- Provide an ability to add a custom homepage link to the header, more
details in
[https://github.com/Kotlin/dokka/issues/2948#issuecomment-1976723089](https://togithub.com/Kotlin/dokka/issues/2948#issuecomment-1976723089)
- Fixed tab selection resetting after navigating to a different page
([https://github.com/Kotlin/dokka/issues/2899](https://togithub.com/Kotlin/dokka/issues/2899))
- Fixed inline code not always being aligned with the surrounding text
([https://github.com/Kotlin/dokka/issues/3228](https://togithub.com/Kotlin/dokka/issues/3228))
- Fixed the "No options found" text in search being barely visible
([https://github.com/Kotlin/dokka/issues/3281](https://togithub.com/Kotlin/dokka/issues/3281))
- Fixed empty HTML tags being rendered for no reason
([https://github.com/Kotlin/dokka/pull/3343](https://togithub.com/Kotlin/dokka/pull/3343),
[https://github.com/Kotlin/dokka/issues/3095](https://togithub.com/Kotlin/dokka/issues/3095))

#### Runners

##### Gradle Plugin

- Mark tasks as not compatible with Gradle configuration cache, second
try
([https://github.com/Kotlin/dokka/pull/3438](https://togithub.com/Kotlin/dokka/pull/3438)).
Thanks to [@&#8203;3flex](https://togithub.com/3flex) for noticing and
fixing the problem!

##### Maven Plugin

- Fixed `dokka:help` being absent
([https://github.com/Kotlin/dokka/issues/3035](https://togithub.com/Kotlin/dokka/issues/3035)).
Thanks to [@&#8203;aSemy](https://togithub.com/aSemy)!
- Fixed the source links configuration not working
([https://github.com/Kotlin/dokka/pull/3046](https://togithub.com/Kotlin/dokka/pull/3046)).
Thanks to [@&#8203;freya022](https://togithub.com/freya022) for fixing
this one!

##### CLI runner

- Allow using relative paths in the `sourceRoots` configuration option
([https://github.com/Kotlin/dokka/issues/2571](https://togithub.com/Kotlin/dokka/issues/2571))

#### Plugin API

- Provide an extension point to customize the rendering of code blocks
in HTML format
([https://github.com/Kotlin/dokka/issues/3244](https://togithub.com/Kotlin/dokka/issues/3244))

#### Other:

- Make sure `wasm-js` and `wasm-wasi` targets introduced in Kotlin
1.9.20 are supported
([https://github.com/Kotlin/dokka/issues/3310](https://togithub.com/Kotlin/dokka/issues/3310))
- Avoid concurrent invocations of Kotlin compiler's API due to the
compiler API itself not always being thread safe
([https://github.com/Kotlin/dokka/issues/3151](https://togithub.com/Kotlin/dokka/issues/3151)).
No noticeable performance loss is expected.
- Bump dependencies to the latest versions
([https://github.com/Kotlin/dokka/pull/3231](https://togithub.com/Kotlin/dokka/pull/3231),
[https://github.com/Kotlin/dokka/pull/3206](https://togithub.com/Kotlin/dokka/pull/3206),
[https://github.com/Kotlin/dokka/pull/3204](https://togithub.com/Kotlin/dokka/pull/3204))
- Fix a documentation link
([https://github.com/Kotlin/dokka/pull/3213](https://togithub.com/Kotlin/dokka/pull/3213)).
Thanks to [@&#8203;SubhrajyotiSen](https://togithub.com/SubhrajyotiSen)
for noticing and fixing it!
- Various build and project structure improvements
([https://github.com/Kotlin/dokka/pull/3174](https://togithub.com/Kotlin/dokka/pull/3174),
[https://github.com/Kotlin/dokka/issues/3132](https://togithub.com/Kotlin/dokka/issues/3132)).
Enormous thanks to [@&#8203;aSemy](https://togithub.com/aSemy) for the
help!

See [Dokka
1.9.20](https://togithub.com/Kotlin/dokka/milestone/30?closed=1)
milestone for the list of all changes.

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMjcuMSIsInVwZGF0ZWRJblZlciI6IjM3LjIyNy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
github-merge-queue bot referenced this pull request in slackhq/circuit Mar 5, 2024
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [org.jetbrains.dokka](https://togithub.com/Kotlin/dokka) | plugin |
patch | `1.9.10` -> `1.9.20` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>Kotlin/dokka (org.jetbrains.dokka)</summary>

### [`v1.9.20`](https://togithub.com/Kotlin/dokka/releases/tag/v1.9.20):
1.9.20

#### General bugfixes

- Fixed sealed interfaces not having the `sealed` keyword in signatures
([https://github.com/Kotlin/dokka/issues/2994](https://togithub.com/Kotlin/dokka/issues/2994))
- Fixed incorrect links in multi-module projects with non-unique package
names
([https://github.com/Kotlin/dokka/issues/2272](https://togithub.com/Kotlin/dokka/issues/2272)).
Huge thanks to [@&#8203;EddieRingle](https://togithub.com/EddieRingle)!
- Fixed member extensions not being shown on index pages in certain
scenarios
([https://github.com/Kotlin/dokka/issues/3187](https://togithub.com/Kotlin/dokka/issues/3187))
- Fixed Java's inner classes not having the `inner` keyword in Kotlin
signatures
([https://github.com/Kotlin/dokka/issues/2793](https://togithub.com/Kotlin/dokka/issues/2793))
- Fixed Java's `@param` tag not working with type parameters
([https://github.com/Kotlin/dokka/issues/3199](https://togithub.com/Kotlin/dokka/issues/3199))
- Fixed Dokka failing in KMP projects when the JVM source set is
suppressed
([https://github.com/Kotlin/dokka/issues/3209](https://togithub.com/Kotlin/dokka/issues/3209))

#### HTML format

- Provide an ability to add a custom homepage link to the header, more
details in
[https://github.com/Kotlin/dokka/issues/2948#issuecomment-1976723089](https://togithub.com/Kotlin/dokka/issues/2948#issuecomment-1976723089)
- Fixed tab selection resetting after navigating to a different page
([https://github.com/Kotlin/dokka/issues/2899](https://togithub.com/Kotlin/dokka/issues/2899))
- Fixed inline code not always being aligned with the surrounding text
([https://github.com/Kotlin/dokka/issues/3228](https://togithub.com/Kotlin/dokka/issues/3228))
- Fixed the "No options found" text in search being barely visible
([https://github.com/Kotlin/dokka/issues/3281](https://togithub.com/Kotlin/dokka/issues/3281))
- Fixed empty HTML tags being rendered for no reason
([https://github.com/Kotlin/dokka/pull/3343](https://togithub.com/Kotlin/dokka/pull/3343),
[https://github.com/Kotlin/dokka/issues/3095](https://togithub.com/Kotlin/dokka/issues/3095))

#### Runners

##### Gradle Plugin

- Mark tasks as not compatible with Gradle configuration cache, second
try
([https://github.com/Kotlin/dokka/pull/3438](https://togithub.com/Kotlin/dokka/pull/3438)).
Thanks to [@&#8203;3flex](https://togithub.com/3flex) for noticing and
fixing the problem!

##### Maven Plugin

- Fixed `dokka:help` being absent
([https://github.com/Kotlin/dokka/issues/3035](https://togithub.com/Kotlin/dokka/issues/3035)).
Thanks to [@&#8203;aSemy](https://togithub.com/aSemy)!
- Fixed the source links configuration not working
([https://github.com/Kotlin/dokka/pull/3046](https://togithub.com/Kotlin/dokka/pull/3046)).
Thanks to [@&#8203;freya022](https://togithub.com/freya022) for fixing
this one!

##### CLI runner

- Allow using relative paths in the `sourceRoots` configuration option
([https://github.com/Kotlin/dokka/issues/2571](https://togithub.com/Kotlin/dokka/issues/2571))

#### Plugin API

- Provide an extension point to customize the rendering of code blocks
in HTML format
([https://github.com/Kotlin/dokka/issues/3244](https://togithub.com/Kotlin/dokka/issues/3244))

#### Other:

- Make sure `wasm-js` and `wasm-wasi` targets introduced in Kotlin
1.9.20 are supported
([https://github.com/Kotlin/dokka/issues/3310](https://togithub.com/Kotlin/dokka/issues/3310))
- Avoid concurrent invocations of Kotlin compiler's API due to the
compiler API itself not always being thread safe
([https://github.com/Kotlin/dokka/issues/3151](https://togithub.com/Kotlin/dokka/issues/3151)).
No noticeable performance loss is expected.
- Bump dependencies to the latest versions
([https://github.com/Kotlin/dokka/pull/3231](https://togithub.com/Kotlin/dokka/pull/3231),
[https://github.com/Kotlin/dokka/pull/3206](https://togithub.com/Kotlin/dokka/pull/3206),
[https://github.com/Kotlin/dokka/pull/3204](https://togithub.com/Kotlin/dokka/pull/3204))
- Fix a documentation link
([https://github.com/Kotlin/dokka/pull/3213](https://togithub.com/Kotlin/dokka/pull/3213)).
Thanks to [@&#8203;SubhrajyotiSen](https://togithub.com/SubhrajyotiSen)
for noticing and fixing it!
- Various build and project structure improvements
([https://github.com/Kotlin/dokka/pull/3174](https://togithub.com/Kotlin/dokka/pull/3174),
[https://github.com/Kotlin/dokka/issues/3132](https://togithub.com/Kotlin/dokka/issues/3132)).
Enormous thanks to [@&#8203;aSemy](https://togithub.com/aSemy) for the
help!

See [Dokka
1.9.20](https://togithub.com/Kotlin/dokka/milestone/30?closed=1)
milestone for the list of all changes.

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMjcuMSIsInVwZGF0ZWRJblZlciI6IjM3LjIyNy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure Everything related to builds tools, CI configurations and project tooling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor artifact publishing
3 participants