Skip to content

Commit

Permalink
DAT-17531 DevOps: remove FOSSA AI PR comments and add status check (#201
Browse files Browse the repository at this point in the history
)

* ci(workflows): add support for specifying different Maven profiles in GitHub workflows

* build(workflows): update liquibase/build-logic workflows to version v0.7.3 for consistency and bug fixes
docs(README): update release version in README to v0.7.3 for accuracy and clarity

* build(create-release.yml): update liquibase/build-logic version to v0.7.4
build(extension-attach-artifact-release.yml): update liquibase/build-logic version to v0.7.4
build(extension-release-prepare.yml): update liquibase/build-logic version to v0.7.4
build(extension-release-published.yml): update liquibase/build-logic version to v0.7.4
build(fossa_ai.yml): update liquibase/build-logic version to v0.7.4
build(os-extension-test.yml): update liquibase/build-logic version to v0.7.4
build(pom-release-published.yml): update liquibase/build-logic version to v0.7.4
build(pro-extension-test.yml): update liquibase/build-logic version to v0.7.4
docs(README.md): update build-logic version to v0.7.4 in release instructions

* chore(create-release.yml): update sonar-push.yml version to v0.7.5
chore(extension-attach-artifact-release.yml): update build logic scripts to v0.7.5
chore(extension-release-prepare.yml): update extension-release-rollback.yml version to v0.7.5
chore(extension-release-published.yml): update extension-release-prepare.yml version to v0.7.5
chore(fossa_ai.yml): update build logic scripts to v0.7.5 for FOSSA AI detection

ci(os-extension-test.yml): update sonar-pull-request.yml version to v0.7.5
ci(pom-release-published.yml): update extension-release-prepare.yml version to v0.7.5
ci(pro-extension-test.yml): update sonar-pull-request.yml version to v0.7.5
docs(README.md): update build-logic version from v0.7.4 to v0.7.5 for new release

* chore(fossa_ai.yml): simplify conditional expressions for better readability and maintainability

* build(workflows): update workflow files to use version v0.7.4 of build-logic
docs(README): update version reference in README to v0.7.4 for consistency

* chore(fossa_ai.yml): refactor FOSSA Snippets Detection workflow to only decorate PR if '0 matches' are found in snippets.out file

* chore(fossa_ai.yml): refactor FOSSA AI workflow to include detection of AI generated code and decorate PRs accordingly
chore(fossa_ai.yml): refactor FOSSA Test workflow to include skipping PR decoration if no issues found

* build(create-release.yml): update liquibase/build-logic version to v0.7.5
build(extension-attach-artifact-release.yml): update script files to v0.7.5 version
build(extension-release-prepare.yml): update liquibase/build-logic version to v0.7.5
build(extension-release-published.yml): update liquibase/build-logic version to v0.7.5
build(fossa_ai.yml): update fossa configuration to v0.7.5
build(fossa_ai.yml): update fossa scripts to v0.7.5 version
build(fossa_ai.yml): update fossa snippets detection script to v0.7.5
build(fossa_ai.yml): update fossa AI detection script to v0.7.5
build(fossa_ai.yml): update fossa test script to v0.7.5
build(os-extension-test.yml): update liquibase/build-logic version to v0.7.5

ci(pom-release-published.yml): update liquibase/build-logic extension-release-prepare.yml to v0.7.5
ci(pro-extension-test.yml): update liquibase/build-logic sonar-pull-request.yml to v0.7.5
docs(README.md): update build-logic version from v0.7.4 to v0.7.5 for new release

* refactor(fossa_ai.yml): remove duplicate fs require statements to improve code readability and maintainability

* fix(fossa_ai.yml): fix conditional check to create comment only if output does not contain '0 matches'

* chore(fossa_ai.yml): refactor script to improve readability and remove redundant code

* fix(fossa_ai.yml): fix comment syntax issue in the script block to prevent it from being executed

* chore(fossa_ai.yml): refactor GitHub Actions workflow to improve code readability and remove unnecessary conditional checks

* chore(fossa_ai.yml): update FOSSA configuration to use version v0.7.4 instead of v0.7.5
chore(fossa_ai.yml): remove unnecessary FOSSA snippets detection decoration step
feat(fossa_ai.yml): add support for running FOSSA on pull requests for snippets detection
feat(fossa_ai.yml): add support for running FOSSA AI detection on pull requests if AI generated code is detected

* refactor(fossa_ai.yml): remove duplicate fs require statements to improve code readability and maintainability

* refactor(fossa_ai.yml): refactor script to only execute if 'snippets.out' does not contain '0 matches' to improve efficiency and reduce unnecessary executions.

* chore(fossa_ai.yml): refactor script to improve readability and remove duplicate code by moving common logic to the top of the file

* chore(fossa_ai.yml): refactor FOSSA Snippets Detection Decoration workflow to run on job failure and include additional information in the comment output

* chore(fossa_ai.yml): add echo statement to indicate failure in FOSSA Snippets Detection

* fix(fossa_ai.yml): improve error handling in fossa snippets analysis script to provide more informative messages and properly handle file existence and content validation

* chore(fossa_ai.yml): add FOSSA_DECORATION environment variable to control decoration in FOSSA snippets analysis
chore(fossa_ai.yml): update FOSSA Snippets Detection Decoration step condition to check for FOSSA_DECORATION environment variable before running

* fix(fossa_ai.yml): fix conditional statement to correctly check for FOSSA_DECORATION environment variable value before running the action

* chore(fossa_ai.yml): remove unnecessary FOSSA decoration scripts and related logic to simplify the workflow and improve readability.

* fix(fossa_ai.yml): add error handling to check if output files exist and contain expected content to improve workflow reliability and error reporting

* fix(fossa_ai.yml): update condition to include 'AI generated code' in addition to 'GitHub Copilot generated code' for labeling issues generated by AI assistants

* build(fossa_ai.yml): update fossa_ai.yml to fetch the latest version of .fossa.yml file from the liquibase/build-logic repository
fix(fossa_ai.yml): fix syntax error in the script by adding a missing closing parenthesis after 'AI generated code' condition
  • Loading branch information
jandroav authored Apr 25, 2024
1 parent 4943879 commit 0572fd4
Show file tree
Hide file tree
Showing 9 changed files with 36 additions and 112 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:

jobs:
sonar:
uses: liquibase/build-logic/.github/workflows/sonar-push.yml@v0.7.4
uses: liquibase/build-logic/.github/workflows/sonar-push.yml@v0.7.5
secrets: inherit
with:
extraCommand: ${{ inputs.extraCommand }}
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/extension-attach-artifact-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,9 @@ jobs:
- name: Get Reusable Script Files
run: |
curl -o $PWD/.github/get_draft_release.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.4/.github/get_draft_release.sh
curl -o $PWD/.github/sign_artifact.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.4/.github/sign_artifact.sh
curl -o $PWD/.github/upload_asset.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.4/.github/upload_asset.sh
curl -o $PWD/.github/get_draft_release.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.5/.github/get_draft_release.sh
curl -o $PWD/.github/sign_artifact.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.5/.github/sign_artifact.sh
curl -o $PWD/.github/upload_asset.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.5/.github/upload_asset.sh
chmod +x $PWD/.github/get_draft_release.sh
chmod +x $PWD/.github/sign_artifact.sh
chmod +x $PWD/.github/upload_asset.sh
Expand Down Expand Up @@ -180,7 +180,7 @@ jobs:
- name: Get upload_zip.sh Script File
if: inputs.zip == 'true'
run: |
curl -o $PWD/.github/upload_zip.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.4/.github/upload_zip.sh
curl -o $PWD/.github/upload_zip.sh https://raw.githubusercontent.com/liquibase/build-logic/v0.7.5/.github/upload_zip.sh
chmod +x $PWD/.github/upload_zip.sh
- name: Attach Zip File to Draft Release
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/extension-release-prepare.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ jobs:
release-rollback:
needs: prepare-release
if: ${{ always() && contains(needs.*.result, 'failure') }}
uses: liquibase/build-logic/.github/workflows/extension-release-rollback.yml@v0.7.4
uses: liquibase/build-logic/.github/workflows/extension-release-rollback.yml@v0.7.5
secrets: inherit
with:
extraCommand: ${{ inputs.extraCommand }}
2 changes: 1 addition & 1 deletion .github/workflows/extension-release-published.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ on:

jobs:
maven-release:
uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.7.4
uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.7.5
secrets: inherit
with:
extraCommand: ${{ inputs.extraCommand }}
Expand Down
126 changes: 25 additions & 101 deletions .github/workflows/fossa_ai.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
- name: Get Fossa Configuration
if: ${{ inputs.check_ai_generated_code }}
run: |
curl -o $PWD/.github/.fossa.yml https://raw.githubusercontent.com/liquibase/build-logic/v0.7.4/.github/.fossa.yml
curl -o $PWD/.github/.fossa.yml https://raw.githubusercontent.com/liquibase/build-logic/v0.7.5/.github/.fossa.yml
- name: Install FOSSA CLI
run: |
Expand All @@ -47,6 +47,17 @@ jobs:
run: |
# https://github.com/fossas/fossa-cli/blob/master/docs/references/subcommands/snippets/analyze.md
fossa snippets analyze -o snippets 2>&1 | tee snippets.out
FILE="snippets.out"
if [ -f "$FILE" ]; then
if grep -q "0 matches" "$FILE"; then
echo "The file '$FILE' contains '0 matches'."
else
echo "The file '$FILE' does not contain '0 matches'."
exit 1
fi
else
echo "Error: The file '$FILE' does not exist."
fi
- name: Generate Snippet Dependencies
if: ${{ inputs.check_snippets }}
Expand Down Expand Up @@ -79,104 +90,17 @@ jobs:
run: |
# https://github.com/fossas/fossa-cli/tree/master/docs/references/subcommands/test
fossa test 2>&1 | tee test.out
- name: FOSSA Snippets Detection Decoration
if: github.event_name == 'pull_request' && ${{ inputs.check_snippets }}
uses: actions/github-script@v6
continue-on-error: true
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const run_url = process.env.GITHUB_SERVER_URL + '/' + process.env.GITHUB_REPOSITORY + '/actions/runs/' + process.env.GITHUB_RUN_ID
const run_link = '<a href="' + run_url + '">Actions</a>.'
const fs = require('fs')
const snippets_file = fs.readFileSync('snippets.out', 'utf8')
const snippets = snippets_file.length > 65000 ? snippets_file.toString().substring(0, 65000) + " ..." : snippets_file
const truncated_message = snippets_file.length > 65000 ? "Output is too long and was truncated. You can read full log in " + run_link + "<br /><br />" : ""
const output = `#### FOSSA Snippets Detection 🤖\`${{ steps.snippets.outcome }}\`
<details><summary>Snippets Detection Output</summary>
\`\`\`\n
${snippets}
\`\`\`
</details>
${truncated_message}
*Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ github.workspace }}\`, Workflow: \`${{ github.workflow }}\`*`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
- name: FOSSA Test Decoration
if: github.event_name == 'pull_request'
uses: actions/github-script@v6
continue-on-error: true
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const run_url = process.env.GITHUB_SERVER_URL + '/' + process.env.GITHUB_REPOSITORY + '/actions/runs/' + process.env.GITHUB_RUN_ID
const run_link = '<a href="' + run_url + '">Actions</a>.'
const fs = require('fs')
const test_file = fs.readFileSync('test.out', 'utf8')
const test = test_file.length > 65000 ? test_file.toString().substring(0, 65000) + " ..." : test_file
const truncated_message = test_file.length > 65000 ? "Output is too long and was truncated. You can read full log in " + run_link + "<br /><br />" : ""
const output = `#### FOSSA Test 🧪\`${{ steps.test.outcome }}\`
<details><summary>Test Output</summary>
\`\`\`\n
${test}
\`\`\`
</details>
${truncated_message}
*Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ github.workspace }}\`, Workflow: \`${{ github.workflow }}\`*`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
- name: FOSSA AI Detection Decoration
if: github.event_name == 'pull_request' && ${{ inputs.check_ai_generated_code }}
uses: actions/github-script@v6
continue-on-error: true
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const run_url = process.env.GITHUB_SERVER_URL + '/' + process.env.GITHUB_REPOSITORY + '/actions/runs/' + process.env.GITHUB_RUN_ID
const run_link = '<a href="' + run_url + '">Actions</a>.'
const fs = require('fs')
const analyze_file = fs.readFileSync('analyze.out', 'utf8')
const analyze = analyze_file.length > 65000 ? analyze_file.toString().substring(0, 65000) + " ..." : analyze_file
const truncated_message = analyze_file.length > 65000 ? "Output is too long and was truncated. You can read full log in " + run_link + "<br /><br />" : ""
const output = `#### FOSSA AI Detection 🤖\`${{ steps.analyze-ai.outcome }}\`
<details><summary>AI detection Output</summary>
\`\`\`\n
${analyze}
\`\`\`
</details>
${truncated_message}
*Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ github.workspace }}\`, Workflow: \`${{ github.workflow }}\`*`;
if (analyze_file.includes('GitHub Copilot generated code')) {
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
}
FILE="test.out"
if [ -f "$FILE" ]; then
if grep -q "Test passed" "$FILE"; then
echo "The file '$FILE' contains 'Test passed'."
else
echo "The file '$FILE' does not contain 'Test passed'."
exit 1
fi
else
echo "Error: The file '$FILE' does not exist."
fi
- name: Label PR with AI label
if: ${{ inputs.check_ai_generated_code }}
Expand All @@ -187,7 +111,7 @@ jobs:
script: |
const fs = require('fs')
const analyze_file = fs.readFileSync('analyze.out', 'utf8')
if (analyze_file.includes('GitHub Copilot generated code')) {
if (analyze_file.includes('GitHub Copilot generated code') || analyze_file.includes('AI generated code')) {
github.rest.issues.addLabels({
issue_number: context.issue.number,
owner: context.repo.owner,
Expand Down Expand Up @@ -215,4 +139,4 @@ jobs:
uses: actions/upload-artifact@v3
with:
name: sbom
path: sbom.html
path: sbom.html
2 changes: 1 addition & 1 deletion .github/workflows/os-extension-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ jobs:
sonar-pr:
if: ${{ !inputs.nightly }}
needs: [unit-test]
uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.7.4
uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.7.5
secrets: inherit
with:
extraCommand: ${{ inputs.extraCommand }}
2 changes: 1 addition & 1 deletion .github/workflows/pom-release-published.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,5 +68,5 @@ jobs:
maven-release:
needs: release
uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.7.4
uses: liquibase/build-logic/.github/workflows/extension-release-prepare.yml@v0.7.5
secrets: inherit
2 changes: 1 addition & 1 deletion .github/workflows/pro-extension-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ jobs:
sonar-pr:
if: ${{ !inputs.nightly }}
needs: [unit-test]
uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.7.4
uses: liquibase/build-logic/.github/workflows/sonar-pull-request.yml@v0.7.5
secrets: inherit
with:
extraCommand: ${{ inputs.extraCommand }}
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,4 +451,4 @@ Here the modules we want to generate and aggregate test reports must be specifie

### Releasing a new version of build-logic

When you want to release new version of `build-logic`, it is important to update all the occurrences of previous version eg: `v0.6.9` with the new version eg : `v0.7.4` in all the files. As, the code for the new version internally refers to the old version.
When you want to release new version of `build-logic`, it is important to update all the occurrences of previous version eg: `v0.6.9` with the new version eg : `v0.7.5` in all the files. As, the code for the new version internally refers to the old version.

0 comments on commit 0572fd4

Please sign in to comment.