Tear it all apart... (#197)

The high-level summary is that this replaces all the git log parsing for
issue numbers with calls to the Search and Commit endpoints in the
GitHub REST API. The Search endpoints are used to find the issues and
PRs linked to a specific milestone. The Commit endpoint is used to find
the unique commit authors (as opposed to the creator and/or assignees of
issues and PRs). So, a lot of code was outright removed, a lot was
replaced, and a lot of things were renamed to better align with that
they are. Hopefully, this makes things more clear. Also, while adding
and replacing code, tests have been added so the overall coverage is now
closes to 90%. The only major thing not tested is in the App class.

Major renames and refactorings:

* ChangelogGeneratorMain renamed to App
* GenerateChangelog renamed to ChangelogGenerator
* Merge RepoHostConfig and RepoConfig, with the final result in
RepoConfig
* GitHubListFetcher renamed to GitHubPagingHelper, and refactored as a
helper class for pagination (not specific to issues)
* Rename things that have GitHub in their name to have a capital H
(i.e., GitHub not Github)

Additions/Changes:

* Add GitHubCategoryFinder as a utility to find a category given a list
of labels; this was extracted from GithubTicketFetcher, which has been
deleted
* Add GitHubChange to represent the information from an issue or PR that
we need for the change log
* Make outputFile in ChangelogConfig have a null default value
* Tests and a bunch of test resources (JSON sample responses)
* Enhance ConfigHelpers#externalConfig to use a kiwi
LoggingDeserializationProblemHandler to log deserialization errors (for
example if there's something unexpected in a YAML config file) at WARN
level. There is a TODO to find a better way to report these errors to a
user, rather than an ugly log message.
* Add a milestone function to RepoConfig which returns an
explicitly-provided milestone, or else assumes the revision is in format
vX.Y.Z and removes the leading "v"
* Add several handy-dandy extension methods to MapExtensions
* Update GitHubApi#GitHubResponse#from method to allow for the (odd)
possibility of seeing a negative duration for the rate limit reset. Yes,
it happened to me which is why I added the code...
* Change the println in GitHubApi to DEBUG-level logging
* Add new tests for things that did not have a test, e.g., add AppTest
for App

Command line args:

* Remove working directory ( -w / --working-dir )
* Remove "alwaysIncludePRsFrom" ( -i / --include-prs-from )
* Change --milestone-to-close to just --milestone (short arg remains -M)
* Make previous revision required
* Remove default from revision (was HEAD) and made it required

Removals:

* GitCommit
* GitLogProvider
* Ticket
* TicketParser
* RepoHostConfig (merged into RepoConfig)

Misc:

* Add MockWebServerExtension JUnit Jupiter extension to start/stop a
MockWebServer
* Update several tests to use MockWebServerExtension
* Refactored internals of App#main so that we can test it without
System.exit being called. Did this by extracting an internal "execute"
method which returns an AppResult containing the exit code and the App
itself, which allows us to test the argument parsing. App#main calls
execute and then calls exitProcess with the AppResult.exitCode

Documentation

* Update the main README to reflect all the changes to usage
* Update the documentation in App for the top-level description and
several argument
* Remove the "alwaysIncludePRsFrom" property from the sample kiwi
changelog (sample-kiwi-changelog.yml)

Closes #194

README.md

@@ -14,7 +14,7 @@ Generates changelogs between two versions on GitHub.
 
 You can install using an installation script or by manually building the JAR. The following sections describe these options.
 
-_For both installation types, the script assumes that Maven is available, since it is required to build the JAR file._
+_For both installation types, the script assumes that Maven is available since it is required to build the JAR file._
 
 ### Install directly from GitHub using curl
 
@@ -34,13 +34,15 @@ curl -s  https://raw.githubusercontent.com/kiwiproject/kiwiproject-changelog/mai
 
 Commands that require user confirmation (such as when uninstalling or if you attempt to update an existing installation)
 work using a `curl`-based installation, but you must use the `-y` (yes) option to avoid interactive prompts and
-auto-confirm. To uninstall you can:
+auto-confirm.
+
+To uninstall, you can:
 
 ```shell
 curl -s  https://raw.githubusercontent.com/kiwiproject/kiwiproject-changelog/main/etc/install/install.sh | bash -s -- -u -y
 ```
 
-To install the latest version and overwrite an existing installation, just run the original curl command again with `-y`:
+To install the latest version and overwrite an existing installation, run the original curl command again with `-y`:
 
 ```shell
 curl -s  https://raw.githubusercontent.com/kiwiproject/kiwiproject-changelog/main/etc/install/install.sh | bash -s -- -y
@@ -75,7 +77,8 @@ You can also _uninstall_ using the `-u` flag. You will be prompted for confirmat
 
 If you prefer a manual installation, first clone the repository.
 
-In order to run the changelog generator from the command line, you need to build the _shaded_ JAR which includes all dependencies.
+To run the changelog generator from the command line,
+you need to build the _shaded_ JAR which includes all dependencies.
 
 The default Maven build does not include dependencies, so you need to run the build using the `shaded-jar` profile:
 
@@ -84,7 +87,7 @@ $ mvn package -DskipTests -Pshaded-jar
 ```
 
 This will build the current SNAPSHOT build in the POM. If you want to build a specific version,
-for example the latest production release, just check out the corresponding release tag build the JAR:
+maybe the latest production release, check out the corresponding release tag and build the JAR:
 
 ```bash
 $ git checkout <release-tag>
@@ -102,23 +105,61 @@ Now you can run the JAR file with no arguments (or `-h` / `--help` ) to get usag
 $ java -jar <path-to-jar>/changelog-generator-0.6.0.jar --help
 ```
 
+## How it works
+
+Changelogs are generated by finding issues and pull requests that are linked to a specific milestone.
+The milestone is derived from the revision (tag), which by default is expected to begin with `"v"`.
+For example, revision `v1.4.2` corresponds to milestone `1.4.2` in the GitHub repository.
+
+Revisions can also include pre-release or build metadata, for example `v1.0.0-alpha.1` corresponds
+to milestone `1.0.0-alpha.1`.
+
+If the repository uses a different tag format, you need to specify the milestone explicitly.
+For example, if tags use a format like `rel-1.4.2` then you can still generate the changelog
+as long as you specify the milestone.
+
+Both the revision and the previous revision must be specified. The reason is so the
+changelog generator can find the unique _commit_ authors between these two revisions,
+and list them as contributors to the release. Note specifically that the contributors are
+the people who created the commits, not the people who created the issue or merged the pull
+request. Note also that issues and pull requests do not include any information about
+commits related to them, so the author information can only be obtained from the commits
+that occurred during the milestone.
+
+Labels on the issues and pull requests are mapped to a category.
+For example, the label "enhancement" might map to the category "Improvements".
+And you can also specify an emoji for each category for a nicer-looking changelog.
+
+### How to prevent duplicates in change logs
+
+Because change logs are generated from the issues and pull requests associated with
+a milestone, the resulting change log will contain duplicates if an issue _and its
+associated pull request_ are both linked to the milestone.
+
+To prevent this, link _either_ the issue to a milestone (this is our preference
+for our own repositories), _or_ link the pull request, but _not_ both.
+
+Issues which do not have an associated pull request (was that you who force-pushed 🔥 it
+straight to main?) should be linked to a milestone so that they are included in the change log.
+
+Similarly, pull requests that are not associated with an issue, such as ones created by dependabot,
+should be linked to a milestone so that they are included in the change log.
+
 ## Command line arguments
 
-Here is a sample argument list that generates a changelog for the [kiwi](https://github.com/kiwiproject/kiwi) 
+Here is a sample argument list that generates a changelog for the [kiwi](https://github.com/kiwiproject/kiwi)
 project (inside the kiwiproject organization on GitHub) for version 2.5.0.
-It creates the changelog comparing from revision v2.4.0 to v2.5.0 using the given working directory.
+It creates the changelog comparing from revision v2.4.0 to v2.5.0.
 It also maps several labels to categories, e.g., `-m bug:Bugs` maps from the GitHub label `bugs` to the
 changelog category "Bugs" (you can also use `--mapping`). The `-O` option (you can also use `--category-order`)
-specifies the order of the categories in the generated changelog. And last, the `--include-prs-from` (or `-i`)
-option lets you include PRs from specific users, e.g. a bot like dependabot.
+specifies the order of the categories in the generated changelog.
 
 
 ```
 --repo-host-token [YOUR_GITHUB_ACCESS_TOKEN]
 --repository kiwiproject/kiwi
 --previous-rev v2.4.0
 --revision v2.5.0
---working-dir /home/users/bob/projects/kiwiproject/kiwi
 -m bug:Bugs
 -m "new feature:Improvements"
 -m enhancement:Improvements
@@ -129,7 +170,6 @@ option lets you include PRs from specific users, e.g. a bot like dependabot.
 -O Bugs
 -O Assorted
 -O "Dependency Updates"
---include-prs-from dependabot[bot]
 ```
 
 In the `-m` options, the GitHub label comes first, then a colon, and finally the category name.
@@ -138,11 +178,11 @@ quotes.
 
 Make sure to use the _exact_ same category name in the `-m` and `-O` arguments.
 
-The changelog that was produced using the above arguments is
+The changelog produced using the above arguments is
 [here](https://github.com/kiwiproject/kiwi/releases/tag/v2.5.0).
 
 Note that you need a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)
-that has appropriate access to be able to read issues in the repository. 
+that has appropriate access to be able to read issues in the repository.
 You can supply this token using the command line (as shown above), or by setting an environment variable
 named `KIWI_CHANGELOG_TOKEN`.
 
@@ -176,10 +216,9 @@ For example:
 -O Bugs
 -O Assorted
 -O "Dependency Updates"
---include-prs-from dependabot[bot]
 ```
 
-Make sure to use the same _exact_ same category names in the `-m`, `-e`, and `-O` options.
+Make sure to use the same _exact_ category names in the `-m`, `-e`, and `-O` options.
 
 ## External configuration
 
@@ -213,9 +252,6 @@ categories:
     emoji: ⬆️
     labels:
       - dependencies
-
-alwaysIncludePRsFrom:
-  - dependabot[bot]
 ```
 
 Here is another [sample](sample-kiwi-changelog.yml) changelog configuration. This is the one
@@ -232,11 +268,34 @@ where the changelog generator JAR is executed):
 2. parent directory
 3. user's home directory
 
-This order lets you have a custom changelog configuration for a repository, or a group of 
+This order lets you have a custom changelog configuration for a repository, or a group of
 repositories such as an organization, or a configuration to be used for _any_ repository.
 
 If you don't want any `.kiwi-changelog.yml` configuration files to be used at all, you can
 use the `--ignore-config-files` (or `-g`) command line flag.
 
 You can also override the above and specify a custom configuration file using the
 `--config-file` (or `-n`) command line option, e.g. `--config-file /path/to/custom/changelog.yml`.
+
+If you export `KIWI_CHANGELOG_TOKEN` to your environment, and you use a configuration file, then
+generating change logs can be as simple as:
+
+```shell
+.generate-kiwi-changelog --repository kiwiproject/kiwiproject-changelog \
+    --previous-rev v0.11.0 \
+    --revision v0.12.0 \
+    --output-type GITHUB \
+    --close-milestone \
+    --create-next-milestone 0.13.0
+```
+
+This generates the change log for the `kiwiproject-changelog` repository for milestone `0.12.0`.
+It posts the change log to GitHub, closes the `0.12.0` milestone, and creates the next milestone, `0.13.0`.
+
+Using short argument names, it can be even shorter:
+
+```shell
+.generate-kiwi-changelog -r kiwiproject/kiwiproject-changelog -p v0.11.0 -R v0.12.0 -o GITHUB -C -N 0.13.0
+```
+
+Happy change log generating! 🤪

etc/install/README.txt

@@ -3,7 +3,7 @@ which performs the actual work.
 
 The source repository is: https://github.com/kiwiproject/kiwiproject-changelog/
 
-The default script is named .generate-kiwi-changelog so you need to
+The default script is named .generate-kiwi-changelog, so you need to
 use ls -a to see it (unless the name was changed during installation).
 
 Run the script with `-h` flag to see usage information.

pom.xml

@@ -290,7 +290,7 @@
                         <configuration>
                             <transformers>
                                 <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
-                                    <mainClass>org.kiwiproject.changelog.ChangelogGeneratorMain</mainClass>
+                                    <mainClass>org.kiwiproject.changelog.App</mainClass>
                                 </transformer>
                             </transformers>
                         </configuration>

sample-kiwi-changelog.yml

@@ -46,6 +46,3 @@ categories:
     labels:
       - infrastructure
       - github_actions
-
-alwaysIncludePRsFrom:
-  - dependabot[bot]