Add SARIF as a reporter option #166

shiina4119 · 2024-08-23T17:44:15Z

Fixes #32

shiina4119 · 2024-08-23T17:47:06Z

Testing locally on my machine, this is the SARIF report that's generated. Please LMK if this is fine.

Also please tell me if I need to add more tests.

Lastly I am new to Go, so please forgive me if there are any obvious mistakes.

{
  "version": "2.1.0",
  "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.4.json",
  "runs": [
    {
      "tool": {
        "driver": {
          "name": "config-file-validator",
          "informationUri": "https://github.com/Boeing/config-file-validator/issues/32"
        }
      },
      "artifacts": [
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.csv"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.editorconfig"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.env"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.hcl"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.hocon"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.ini"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.json"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.plist"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.properties"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.toml"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/bad.yml"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.csv"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.editorconfig"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.env"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.hcl"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.hocon"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.ini"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.json"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.plist"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.properties"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.toml"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/good.yaml"
          }
        },
        {
          "location": {
            "uri": "/home/shiina/projects/cfv-test/result.xml"
          }
        }
      ],
      "results": [
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "parse error on line 1, column 20: bare \" in non-quoted-field"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.csv",
                  "index": 0
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "cannot load ini file: unclosed section: [*.md\n"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.editorconfig",
                  "index": 1
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "Error at line 2: invalid escape sequence: \"a\""
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.env",
                  "index": 2
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "error at line 1 column 1: :1,1-2: Invalid argument name; Argument names must not be quoted."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.hcl",
                  "index": 3
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "two adjacent commas at: 3:16, adjacent commas in arrays and objects are invalid!"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.hocon",
                  "index": 4
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "key-value delimiter not found: name value\n"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.ini",
                  "index": 5
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "Error at line 3 column 14: invalid character ':' after array element"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.json",
                  "index": 6
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "plist: error parsing XML property list: missing value in dictionary"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.plist",
                  "index": 7
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "circular reference in:\nkey=${key}\n"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.properties",
                  "index": 8
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "Error at line 6 column 6: toml: expected character ="
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.toml",
                  "index": 9
                }
              }
            }
          ]
        },
        {
          "kind": "fail",
          "level": "error",
          "message": {
            "text": "yaml: line 3: did not find expected '-' indicator"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/bad.yml",
                  "index": 10
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.csv",
                  "index": 11
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.editorconfig",
                  "index": 12
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.env",
                  "index": 13
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.hcl",
                  "index": 14
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.hocon",
                  "index": 15
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.ini",
                  "index": 16
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.json",
                  "index": 17
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.plist",
                  "index": 18
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.properties",
                  "index": 19
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.toml",
                  "index": 20
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/good.yaml",
                  "index": 21
                }
              }
            }
          ]
        },
        {
          "kind": "pass",
          "level": "none",
          "message": {
            "text": "No errors detected"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "/home/shiina/projects/cfv-test/result.xml",
                  "index": 22
                }
              }
            }
          ]
        }
      ]
    }
  ]
}

shiina4119 · 2024-10-03T20:24:20Z

Hey, would appreciate it if someone reviewed the PR :)

kehoecj

The Sarif is showing as invalid. Please run it through the validator tool and fix the identified issues. If some of the findings are not applicable please explain.

shiina4119 · 2024-10-11T16:39:18Z

Some changes I made:

Updated the $schema field. It now points to the latest schema.
Added the version field in the driver object.
Removed the artifacts object as the validator says its unnecessary.
appended file:/// in front of the uri field in the artifactLocation object.

The validation is still failing because the ruleId field is missing from the elements in the results array. IMO this field would have been useful if we were doing schema validation but for syntax check, I don't know what to put here.

kehoecj · 2024-10-21T17:45:44Z

The validation is still failing because the ruleId field is missing from the elements in the results array. IMO this field would have been useful if we were doing schema validation but for syntax check, I don't know what to put here.

I think we can leave it blank for now and not pass that part of the validation. I'll write up a story for adding that in as we support schema validation

cmd/validator/validator.go

pkg/reporter/sarif_reporter.go

cmd/validator/validator.go: Add option to select reporter type as SARIF. pkg/reporter/sarif_reporter.go: Create SARIF report. https://sarifweb.azurewebsites.net/

cmd/validator/validator_test.go: Test for --reporter=sarif flag. pkg/reporter/reporter_test.go: Test for SARIF report.

… outside.

kehoecj

LGTM - thanks for the PR @shiina4119

shiina4119 changed the title ~~Add support for SARIF reporter~~ Add support for SARIF reports Aug 23, 2024

shiina4119 changed the title ~~Add support for SARIF reports~~ Add SARIF as a reporter option Aug 23, 2024

kehoecj self-requested a review August 24, 2024 02:08

shiina4119 force-pushed the sarif-reporter branch 2 times, most recently from 1567a6b to 7bf8532 Compare August 24, 2024 14:15

kehoecj added the waiting-on-maintainer-review PR is waiting to be reviewed and functionally tested by the maintainers label Aug 26, 2024

shiina4119 force-pushed the sarif-reporter branch from 0087232 to c8db06e Compare September 3, 2024 10:41

kehoecj mentioned this pull request Oct 2, 2024

Added support for sarif reports #108

Closed

kehoecj added the hacktoberfest 🎃 Hacktoberfest 2024 label Oct 2, 2024

kehoecj reviewed Oct 7, 2024

View reviewed changes

kehoecj added pr-action-requested PR is awaiting feedback from the submitting developer and removed waiting-on-maintainer-review PR is waiting to be reviewed and functionally tested by the maintainers labels Oct 7, 2024

shiina4119 force-pushed the sarif-reporter branch from a707db3 to dc1fa23 Compare October 11, 2024 17:01

shiina4119 requested a review from kehoecj October 11, 2024 19:46

kehoecj added waiting-on-maintainer-review PR is waiting to be reviewed and functionally tested by the maintainers and removed pr-action-requested PR is awaiting feedback from the submitting developer labels Oct 14, 2024

kehoecj reviewed Oct 21, 2024

View reviewed changes

cmd/validator/validator.go Outdated Show resolved Hide resolved

kehoecj reviewed Oct 21, 2024

View reviewed changes

pkg/reporter/sarif_reporter.go Outdated Show resolved Hide resolved

kehoecj reviewed Oct 21, 2024

View reviewed changes

pkg/reporter/sarif_reporter.go Outdated Show resolved Hide resolved

kehoecj added pr-action-requested PR is awaiting feedback from the submitting developer and removed waiting-on-maintainer-review PR is waiting to be reviewed and functionally tested by the maintainers labels Oct 21, 2024

shiina4119 added 5 commits October 22, 2024 19:16

cmd/validator/validator_test.go: Fix typo.

e07e7da

Add SARIF reporter

29672bc

cmd/validator/validator.go: Add option to select reporter type as SARIF. pkg/reporter/sarif_reporter.go: Create SARIF report. https://sarifweb.azurewebsites.net/

Add tests for SARIF reporter.

46d885c

cmd/validator/validator_test.go: Test for --reporter=sarif flag. pkg/reporter/reporter_test.go: Test for SARIF report.

Add test for SARIF reporter writer.

28a83c5

cmd/validator/validator.go: Satisfy goreportcard.

c22bd26

shiina4119 and others added 4 commits October 22, 2024 19:16

Fix SARIF validation errors

4a1cbd5

Combine groupOutput check for junit and sarif.

bdd60c5

Add groupby test for sarif reporter.

67bae28

make version, schema, driver name and infoUri constants and move them…

2aa0c34

… outside.

shiina4119 force-pushed the sarif-reporter branch from dc1fa23 to 2aa0c34 Compare October 22, 2024 19:20

kehoecj added hacktoberfest-accepted Valid PR Hacktoberfest PR and removed pr-action-requested PR is awaiting feedback from the submitting developer labels Oct 22, 2024

kehoecj approved these changes Oct 22, 2024

View reviewed changes

Make gocyclo happy

6a550c8

kehoecj merged commit 5193c9c into Boeing:main Oct 23, 2024
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add SARIF as a reporter option #166

Add SARIF as a reporter option #166

shiina4119 commented Aug 23, 2024

shiina4119 commented Aug 23, 2024

shiina4119 commented Oct 3, 2024

kehoecj left a comment •

edited

Loading

shiina4119 commented Oct 11, 2024 •

edited

Loading

kehoecj commented Oct 21, 2024

kehoecj left a comment

Add SARIF as a reporter option #166

Add SARIF as a reporter option #166

Conversation

shiina4119 commented Aug 23, 2024

shiina4119 commented Aug 23, 2024

shiina4119 commented Oct 3, 2024

kehoecj left a comment • edited Loading

Choose a reason for hiding this comment

shiina4119 commented Oct 11, 2024 • edited Loading

kehoecj commented Oct 21, 2024

kehoecj left a comment

Choose a reason for hiding this comment

kehoecj left a comment •

edited

Loading

shiina4119 commented Oct 11, 2024 •

edited

Loading