You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
IgnatBeresnev opened this issue
Aug 24, 2023
· 0 comments
Labels
tech-debtA technical issue that is not observable by the users, but improves maintainers quality of lifetopic: K2Issues / PRs that are related to the K2 migration. See #2888
Dokka has good unit test coverage (> 80%) and OK-ish integration tests, but sometimes dependency updates or internal refactorings can introduce bugs or barely noticeable changes in behaviour or in the output. Most of the time, it's insignificant and not even bugs, but it would not hurt to know about it in advance.
Proposal
To make sure such subtle changes do not go unnoticed, it should be possible to compare the output of the "old" version of Dokka (before the update/refactoring) and the "new" (after). If there's no regression, the diffs should be the same. If even the smallest detail changes, the diff should show it.
This will help immensely with testing Dokka's migration to the K2 compiler (#2888) and with testing the new Gradle runner (#3131).
What exactly should be compared (the HTML outputs or the serialized documentable model), and how, is to be discussed.
Notes
There are projects that already have similar tests and that we can learn from:
Dackka (Google's Dokka plugin for Android docs) compares the diffs of their own HTML format.
Dokkatoo compares the HTML output that Dokka's native Gradle plugin produces with its own, but only for example projects. Ideally, it would be nice to compare big user projects used in integration tests.
tech-debtA technical issue that is not observable by the users, but improves maintainers quality of lifetopic: K2Issues / PRs that are related to the K2 migration. See #2888
Background
Dokka has good unit test coverage (> 80%) and OK-ish integration tests, but sometimes dependency updates or internal refactorings can introduce bugs or barely noticeable changes in behaviour or in the output. Most of the time, it's insignificant and not even bugs, but it would not hurt to know about it in advance.
Proposal
To make sure such subtle changes do not go unnoticed, it should be possible to compare the output of the "old" version of Dokka (before the update/refactoring) and the "new" (after). If there's no regression, the diffs should be the same. If even the smallest detail changes, the diff should show it.
This will help immensely with testing Dokka's migration to the K2 compiler (#2888) and with testing the new Gradle runner (#3131).
What exactly should be compared (the HTML outputs or the serialized documentable model), and how, is to be discussed.
Notes
There are projects that already have similar tests and that we can learn from:
Related issues: #2925
The text was updated successfully, but these errors were encountered: