Skip to content

Commit

Permalink
1.4.0 integration test fixes (#2842)
Browse files Browse the repository at this point in the history
* Fix typo

* Update release testing
  • Loading branch information
paul-dingemans authored Oct 24, 2024
1 parent 90413e7 commit 8718f5a
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 33 deletions.
44 changes: 21 additions & 23 deletions RELEASE_TESTING.MD
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ Before releasing a new version of KtLint, the release candidate is tested on a s
```shell
# Examples of usage
#
# Do not run ktlint commands with this script. Instead run them direct from the `sample-projects` directory
# ktlint --version && ktlint "**/*.kt" -v --relative --format --force-lint-after-format
#
# Outstanding changes per project
# ./exec-in-each-project.sh "git status"
#
# Rollback outstanding changes per project
# ./exec-in-each-project.sh "git reset --hard"
#
# Run ktlint standard rules
# ktlint --version && ktlint "**/*.kt" -v --relative -F
#
# Commit changes of standard rules:
# ./exec-in-each-project.sh "git commit -m \"ktlint (0.43.2) -F\""
Expand All @@ -40,12 +40,12 @@ Before releasing a new version of KtLint, the release candidate is tested on a s
```
4. Create an alias or script to run the latest released version of ktlint (note that this script will print the version as reference to which version is used):
```shell
alias ktlint-prev="ktlint-1.3.0 $@" # Replace with the latest release version
alias ktlint-prev="ktlint-1.3.1 $@" # Replace with the latest release version
```
Note that `~/git/ktlint` is the directory in which the ktlint project is checked out and that `~/git/ktlint/ktlint` refers to the `ktlint` CLI-module.
5. Create an alias or script to run the latest development-version of ktlint (note that this script will print the version and the datetime of compilation as reference to which version is used):
```shell
alias ktlint-dev="echo 'ktlint-dev version:';ls -aslh ~/git/ktlint/pinterest/ktlint/ktlint-cli/build/run/ktlint;echo ' ';~/git/ktlint/pinterest/ktlint/ktlint-cli/build/run/ktlint $@"
alias ktlint-dev="echo 'ktlint-dev version:';ls -aslh ~/git/ktlint/ktlint/ktlint-cli/build/run/ktlint;echo ' ';~/git/ktlint/ktlint/ktlint-cli/build/run/ktlint $@"
```
Note that `~/git/ktlint/pinterest/ktlint` is the directory in which the ktlint project is checked out.

Expand Down Expand Up @@ -82,7 +82,6 @@ Formatting projects in which ktlint is not used may result in a huge amount of f
[graphql-kotlin/**/*.{kt,kts}]
ktlint_standard_no-unused-imports = disabled
ktlint_standard_import-ordering = disabled
[kotlinx.coroutines/**/*.{kt,kts}]
ktlint_standard_no-wildcard-imports = disabled
Expand All @@ -92,7 +91,6 @@ Formatting projects in which ktlint is not used may result in a huge amount of f
# be changed:
# 1) Disable the "root = true" property so that each project ultimately falls back on this file. In this way offending
# rules can be easily enabled/disabled for all test projects
# 2) From the graphql project remove the `disabled_rules` property
ktlint_code_style = ktlint_official
ktlint_experimental = enabled
ktlint_standard = enabled
Expand All @@ -109,7 +107,7 @@ Formatting projects in which ktlint is not used may result in a huge amount of f
Note: Ignore all output as this is the old version!
4. Format the sample projects with the previous (*latest released*) ktlint version:
```shell
ktlint-prev --format --force-lint-after-format --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script.
ktlint-prev -F --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script.
```
Note: Ignore all output as this is the old version!
5. Commit changes:
Expand All @@ -132,25 +130,25 @@ Formatting projects in which ktlint is not used may result in a huge amount of f
ktlint-dev --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want to use the one combined baseline.xml file for all projects.
```
Inspect the output roughly (detailed inspection is done when formatting):
* Is the amount of logging messages comparable to before? If not, are the changes intended?
* Are violations related to rules that have actually been added or changed?
* If you see an error like below, then this version obviously may *not* be released. It is best to fix this error before continuing with testing and validating!
* Is the amount of logging messages comparable to before? If not, are the changes intended?
* Are violations related to rules that have actually been added or changed?
* If you see an error like below, then this version obviously may *not* be released. It is best to fix this error before continuing with testing and validating!
```plain
Internal Error (...) in file '...' at position '0:0. Please create a ticket at https://github.com/pinterest/ktlint/issues ...
```
9. Format with *latest development* version:
```shell
ktlint-dev --format --force-lint-after-format --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want to use the one combined baseline.xml file for all projects.
ktlint-dev -F --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want to use the one combined baseline.xml file for all projects.
```
Inspect the output carefully:
* If you see an error like below, then this version obviously may *not* be released. It is best to fix this error before continuing with testing and validating!
* If you see an error like below, then this version obviously may *not* be released. It is best to fix this error before continuing with testing and validating!
```plain
Internal Error (...) in file '...' at position '0:0. Please create a ticket at https://github.com/pinterest/ktlint/issues ...
```
* Usually it helps to disable all rules that emit violations, except one of those rules. In this way it is possible to evaluate the changes rule by rule.
* Ideally, no violations are shown. This means that all violations have been autocorrected. Note that violations might pop up that previously were suppressed via the baseline. This can happen as due to code changes, the references in the baseline.xml no longer match with the positions where they occur. First check the code changes, and regenerating the baseline before verifying the next can be a helpful approach.
* Violations which could not be autocorrected should be validated for correctness but do not block the release as most likely this is intended behavior.
* If a violation is shown which is not marked as being "can not be autocorrected" this means that during autocorrect of another violation a new violations has been introduced. This should be fixed before releasing especially when the next format introduces the original violation again which of course would result in an endless loop.
* Usually it helps to disable all rules that emit violations, except one of those rules. In this way it is possible to evaluate the changes rule by rule.
* Ideally, no violations are shown. This means that all violations have been autocorrected. Note that violations might pop up that previously were suppressed via the baseline. This can happen as due to code changes, the references in the baseline.xml no longer match with the positions where they occur. First check the code changes, and regenerating the baseline before verifying the next can be a helpful approach.
* Violations which could not be autocorrected should be validated for correctness but do not block the release as most likely this is intended behavior.
* If a violation is shown which is not marked as being "can not be autocorrected" this means that during autocorrect of another violation a new violations has been introduced. This should be fixed before releasing especially when the next format introduces the original violation again which of course would result in an endless loop.
10. Inspect all fixed violations, Of course inspection similar violations tens of times does not make sense. At least check different types of violations a couple of times. Commit changes which do not need to be inspected again:
```shell
./exec-in-each-project.sh "git add --all && git commit -m \"Fixed with latest development version\""
Expand All @@ -165,14 +163,14 @@ Formatting projects in which ktlint is not used may result in a huge amount of f
Internal Error (...) in file '...' at position '0:0. Please create a ticket at https://github.com/pinterest/ktlint/issues ...
```
12. Rerun lint with *latest development* version and verify that no violations are reported:
```shell
ktlint-dev --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want one combined baseline.xml file for all projects.
```
```shell
ktlint-dev --baseline=baseline.xml --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want one combined baseline.xml file for all projects.
```
No violations should be reported in this run.
13. Rerun format with *latest development* version without baseline:
```shell
ktlint-dev --format --force-lint-after-format --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want to use the one combined baseline.xml file for all projects.
```
```shell
ktlint-dev -F --relative # Do not call this command via the "./exec-in-each-project.sh" script as we want to use the one combined baseline.xml file for all projects.
```
As the baseline is removed, thousands of violations are to be expected. Check at least in the summary that no internal errors are thrown like below:
```plain
Internal Error (...) in file '...' at position '0:0. Please create a ticket at https://github.com/pinterest/ktlint/issues ...
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,7 @@ public class BlankLineBetweenWhenConditions :
?.let { whitespaceBeforeWhenEntry ->
emitAndApprove(
whitespaceBeforeWhenEntry.startOffset + 1,
"Add a blank line between all when-condition in case at least one multiline when-condition is found in the " +
"statement",
"Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement",
true,
).ifAutocorrectAllowed {
whitespaceBeforeWhenEntry.upsertWhitespaceBeforeMe("\n${whenEntry.indent()}")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ class BlankLineBetweenWhenConditionsTest {
@Suppress("ktlint:standard:argument-list-wrapping", "ktlint:standard:max-line-length")
blankLineAfterWhenConditionRuleAssertThat(code)
.hasLintViolations(
LintViolation(4, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement"),
LintViolation(6, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement"),
LintViolation(4, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement"),
LintViolation(6, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement"),
).isFormattedAs(formattedCode)
}

Expand Down Expand Up @@ -116,8 +116,8 @@ class BlankLineBetweenWhenConditionsTest {
@Suppress("ktlint:standard:argument-list-wrapping", "ktlint:standard:max-line-length")
blankLineAfterWhenConditionRuleAssertThat(code)
.hasLintViolations(
LintViolation(4, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement"),
LintViolation(5, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement"),
LintViolation(4, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement"),
LintViolation(5, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement"),
).isFormattedAs(formattedCode)
}

Expand Down Expand Up @@ -152,8 +152,8 @@ class BlankLineBetweenWhenConditionsTest {
@Suppress("ktlint:standard:argument-list-wrapping", "ktlint:standard:max-line-length")
blankLineAfterWhenConditionRuleAssertThat(code)
.hasLintViolations(
LintViolation(5, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement"),
LintViolation(7, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement"),
LintViolation(5, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement"),
LintViolation(7, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement"),
).isFormattedAs(formattedCode)
}

Expand Down Expand Up @@ -182,7 +182,7 @@ class BlankLineBetweenWhenConditionsTest {
""".trimIndent()
@Suppress("ktlint:standard:argument-list-wrapping", "ktlint:standard:max-line-length")
blankLineAfterWhenConditionRuleAssertThat(code)
.hasLintViolation(4, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement")
.hasLintViolation(4, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement")
.isFormattedAs(formattedCode)
}

Expand All @@ -209,7 +209,7 @@ class BlankLineBetweenWhenConditionsTest {
""".trimIndent()
@Suppress("ktlint:standard:argument-list-wrapping", "ktlint:standard:max-line-length")
blankLineAfterWhenConditionRuleAssertThat(code)
.hasLintViolation(4, 1, "Add a blank line between all when-condition in case at least one multiline when-condition is found in the statement")
.hasLintViolation(4, 1, "Add a blank line between all when-conditions in case at least one multiline when-condition is found in the statement")
.isFormattedAs(formattedCode)
}
}
Expand Down

0 comments on commit 8718f5a

Please sign in to comment.