Escape newline characters while extracting pr/commit description from github event json.

[github-actions]

Gradle check has failed on push of your commit a6bbc09.
Please examine the workflow log .
Is the failure a flaky test unrelated to your change?

[dblock]

Link missing 🤔

https://github.com/opensearch-project/OpenSearch/actions/runs/8524544204/job/23349365783

a6bbc09

Run repo_url="https://github.com/opensearch-project/OpenSearch"
  repo_url="https://github.com/opensearch-project/OpenSearch"
  ref_id=$(git rev-parse HEAD)
  branch_name=$(git rev-parse --abbrev-ref HEAD)
  echo "event_name=push" >> $GITHUB_ENV
  echo "branch_name=$branch_name" >> $GITHUB_ENV
  echo "pr_from_sha=$ref_id" >> $GITHUB_ENV
  echo "pr_from_clone_url=$repo_url" >> $GITHUB_ENV
  echo "pr_to_clone_url=$repo_url" >> $GITHUB_ENV
  echo "pr_title=Push trigger $branch_name $ref_id $repo_url" >> $GITHUB_ENV
  echo "pr_number=Null" >> $GITHUB_ENV
  echo "pr_owner=$(jq --raw-output '.commits[0].author.username' $GITHUB_EVENT_PATH)" >> $GITHUB_ENV
  echo "pr_or_commit_description=$(jq --raw-output .head_commit.message $GITHUB_EVENT_PATH)" >> $GITHUB_ENV
  shell: /usr/bin/bash -e {0}
Error: Unable to process file command 'env' successfully.
Error: Invalid format '* Add more identifiers to gradle-check job webhook payload
'

Reverting that commit, #13027

cc: @rishabh6788

[dblock]

Closed via #13027.

[rishabh6788]

The issue is happening on echo "pr_or_commit_description=$(jq --raw-output .head_commit.message $GITHUB_EVENT_PATH)" >> $GITHUB_ENV
This is basically adding a new env variable to github workflow runner's env file. Since the variable value contains \n character it is causing the file load error because the new line data will be considered as variable without any assignment.

For e.g., consider the value to be Merge pull request #2 from noobie3007/main\n\nUpdate _config-dev.yml, so when this message is processed it adds the following to the env file:

pr_or_commit_description=Merge pull request #2 from noobie3007/main

Update _config-dev.yml

Now when this file is loaded it will throw an error stating Invalid format 'Update _config-dev.yml'.
One way to avoid this error is use tr to translate all \n to \t.
echo "pr_or_commit_description=$(jq --raw-output .head_commit.message $GITHUB_EVENT_PATH|tr '\n' '\t')" >> $GITHUB_ENV

I have tested this locally and it doesn't throw error.
@dblock Do you know of any better way to handle this?
Also could there be any other edge cases that migh come up while processing PR description or commit messages?

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@rishabh6788 Could you rename this issue to make it clearer what is wrong?

[cwperks]

Another potential way of handling this is to escape newlines before adding to the GITHUB_ENV.

[rishabh6788]

From jq documentation:
--ascii-output / -a: - jq usually outputs non-ASCII Unicode codepoints as UTF-8, even if the input specified them as escape sequences (like "\u03bc"). Using this option, you can force jq to produce pure ASCII output with every non-ASCII character replaced with the equivalent escape sequence.

I tested with this flag and the output was of the format "Merge pull request #2 from noobie3007/main\n\nUpdate _config-dev.yml".

@cwperks Since this prints the actual ascii representation, it doesn't translate \n into a new line while printing output.

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@dblock Thanks for reopening this issue, is there any detail that needs to be added?