adding the script to validate changelog

[nitishchauhan0022]

adding the script to validate changelog by make validate-changelog and updating github action

Checklist

• Changelog has been updated and is aligned with our changelog requirements
• Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #3190

[github-actions]

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

While you are waiting, make sure to:

• Add an entry in our changelog in alphabetical order and link related issue
• Update the documentation, if needed
• Add unit & e2e tests for your changes
• GitHub checks are passing
• Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

• Our contribution guide

[zroubalik]

Thanks for this! The changelog is sorted, but we also tend to put **General**: before all other values.
Also would be nice to add a way to skip the Changelog validation for a certain PR. (eg. we have the skip-e2e label for skipping e2e test check).

[JorTurFer]

Also would be nice to add a way to skip the Changelog validation for a certain PR. (eg. we have the skip-e2e label for skipping e2e test check)

I'm not sure if we should be able to skip Changelog validation, it's more like other CI checks than e2e test IMO, we cannot skip static checks (and this is one static check more). Personally, I'm fine without skipping it, but I prefer to execute it as part of pre-commit if it's possible. If somebody is using pre-commit, it'll cover this new check automatically

WDYT?

[zroubalik]

Also would be nice to add a way to skip the Changelog validation for a certain PR. (eg. we have the skip-e2e label for skipping e2e test check)

I'm not sure if we should be able to skip Changelog validation, it's more like other CI checks than e2e test IMO, we cannot skip static checks (and this is one static check more). Personally, I'm fine without skipping it, but I prefer to execute it as part of pre-commit if it's possible. If somebody is using pre-commit, it'll cover this new check automatically

WDYT?

Good point!

[nitishchauhan0022]

hey @JorTurFer @zroubalik Please take a look.

[nitishchauhan0022]

Thanks for this! The changelog is sorted, but we also tend to put General: before all other values.

@zroubalik To my understanding as there are other values as well checking for General specifically will not give us any result. As we than also need to allow for other **example name** also. If we can limit those names than that check would be a good fit. WDYT?

[JorTurFer]

Thanks for this! The changelog is sorted, but we also tend to put General: before all other values.

@zroubalik To my understanding as there are other values as well checking for General specifically will not give us any result. As we than also need to allow for other example name also. If we can limit those names than that check would be a good fit. WDYT?

General is the only value that has to be prioritized. This is because usually General affects to all the users and that's why we place it on top of all the sections. The style rules are here: https://github.com/kedacore/keda/blob/main/CONTRIBUTING.md#Changelog

Said that, maybe we can add 2 steps in the extract_and_check function, the first step checks for General items and the second one for the rest. WDYT?

[nitishchauhan0022]

@JorTurFer sounds great, i will add it.

[JorTurFer]

This is nice! My only question is if we can check all the changelog and not just unreleased.
I say this because during cherry-picks for hotfix releases we can make mistakes

[nitishchauhan0022]

This is nice! My only question is if we can check all the changelog and not just unreleased. I say this because during cherry-picks for hotfix releases we can make mistakes

we can but we need to change all previous to correct order as well first. Some also have entries missing as well. Like having change name only and doesn't having pr reference.

[JorTurFer]

we can but we need to change all previous to correct order as well first.

If we can, it'd be nice. The changelog has been chaotic in the past 😞

[nitishchauhan0022]

we can but we need to change all previous to correct order as well first.

If we can, it'd be nice. The changelog has been chaotic in the past disappointed

I will try.

[JorTurFer]

This looks nice!
Let me check it with some cases locally and that'd be all I forgot the point of all the versions
Please, remember to undo the change in pr-validation.yaml

[nitishchauhan0022]

hey @JorTurFer I have tried the script to validate the change in each version. The script is:

#!/bin/bash

SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..

# Define filename
filename="$SCRIPT_ROOT/CHANGELOG.md"

# Check if file exists
if [[ ! -f "$filename" ]]; then
    echo "Error: $filename does not exist."
    exit 1
fi

# Storing the version to be checked
mapfile -t versions < <(awk '/## History/{flag=1;next}/## /{flag=0}flag' "$filename" | grep -o '\[[^]]*\]' | sed 's/[][]//g')

# Define a function to extract and sort sections
function extract_and_check() {
  local section=$1
  local content_block=$2
  local content=$(awk "/### $section/{flag=1;next}/### /{flag=0}flag" <<< "$content_block" | grep '^- \*\*')

  # Skip if content does not exist
  if [[ -z "$content" ]]; then
    return
  fi

  # Separate and sort the **General**: lines
  local sorted_general_lines=$(echo "$content" | grep '^- \*\*General\*\*:' | sort)

  # Sort the remaining lines
  local sorted_content=$(echo "$content" | grep -v '^- \*\*General\*\*:' | sort)

  # Check if sorted_general_lines is not empty, then concatenate
  if [[ -n "$sorted_general_lines" ]]; then
      sorted_content=$(printf "%s\n%s" "$sorted_general_lines" "$sorted_content")
  fi

  # Check pattern and throw error if wrong pattern found
  while IFS= read -r line; do
      echo "Error: Wrong pattern found in section: $section , line: $line"
      exit 1
  done < <(grep -Pv '^(-\s\*\*[^*]+\*\*: .*\(\[#(\d+)\]\(https:\/\/github\.com\/kedacore\/(keda|charts)\/(pull|issues)\/\2\)\))$' <<< "$content")

  if [ "$content" != "$sorted_content" ]; then
      echo "Error: Section: $section is not sorted correctly. Correct order:"
      echo "$sorted_content"
      exit 1
  fi
}


# Extract release sections, including "Unreleased", and check them
for version in "${versions[@]}"; do
  release_content=$(awk "/## $version/{flag=1;next}/## v[0-9\.]+/{flag=0}flag" "$filename")


  if [[ -z "$release_content" ]]; then
    echo "No content found for $version Skipping."
    continue
  fi

  echo "Checking section: $version"

  # Separate content into different sections and check sorting for each release
  extract_and_check "New" "$release_content"
  extract_and_check "Improvements" "$release_content"
  extract_and_check "Fixes" "$release_content"
  extract_and_check "Deprecations" "$release_content"
  extract_and_check "Other" "$release_content"

done

But the problem is, some changes are not logged properly, and not easy to search for in the pr and issues, may be they are named differently there.
For example:

Checking section: v2.11.0
Error: Wrong pattern found in section: Improvements , line: - **General**: Enable secret scanning in GitHub repo

[JorTurFer]

But the problem is, some changes are not logged properly, and not easy to search for in the pr and issues, may be they are named differently there.

Let me take a look, we haven't been following the same rules since the beginning, let me check it changelog for trying to limit the scope 😄

[JorTurFer]

There are a lot of issues on the changelog T.T
Can I fix them as part of your PR? As you enabled maintainer to edit the PR, I can commit into your branch if you agree

[JorTurFer]

Based on your script, I have done some small modifications (to exclude v1.* and to support multiple repetitions of the pattern) and I have fixed all the issues on the changelog. After that, it works really nice ❤️

As I said, if you agree, I can commit the changes to your branch directly

[JorTurFer]

[nitishchauhan0022]

@JorTurFer yes please. Thankyou

[JorTurFer]

Done!
Please take a look 🙏 I'm terrible using bash and probably I have done any mistake. I started from your code and update it a bit, so I hope that I didn't do too much shity code xD

[JorTurFer]

/skip-e2e

[nitishchauhan0022]

Done! Please take a look 🙏 I'm terrible using bash and probably I have done any mistake. I started from your code and update it a bit, so I hope that I didn't do too much shity code xD

Thanks for your kind words, It looks good to me.

[nitishchauhan0022]

Why is failing in this:
d6c54d5
Same is running fine locally.

[JorTurFer]

Why is failing in this:
d6c54d5
Same is running fine locally.

That failure that the commit solves is related with missing new line at the end of the file. It's this rule: https://github.com/kedacore/keda/blob/main/.pre-commit-config.yaml#L16C7-L16

About why the script that validates the changelog failed, it looks related with some small differences between grep and sort commands in Windows, Linux and Mac. For example, the grep modifier -P doesn't exist in Mac (I have replaced it with -E) and sort has different default behaviours (in Windows using WSL, default behaviour is case insensitive but in Mac is case sensitive...)

My latest commit updates some lines to be explicit: 2739bb1

Let's see 🤞

[JorTurFer]

Everything is green! 🎉 🎉 🎉 🎉

[JorTurFer]

@zroubalik PTAL