Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate YAML on Bitbucket Server & Refactor the Validation #1846

Merged

Conversation

tricktron
Copy link
Contributor

Changes

The main goal of this pull request is to better surface error logs to the pipeline as code developer on Bitbucket Server: https://issues.redhat.com/browse/SRVKP-6767.

Essentially, this pull request does three things in three commits:

  • Validates the YAML and prints a friendly message for better debugging purposes on bitbucket server as well. Bitbucket Server was not included in Print friendly error message on bad detected yaml #1578.
  • Refactores the whole YAML validation to its own function and uses it in all providers.
  • Adds the bad YAML test for gitea as well so that we have tested it on all providers.

Submitter Checklist

  • 📝 Please ensure your commit message is clear and informative. For guidance on crafting effective commit messages, refer to the How to write a git commit message guide. We prefer the commit message to be included in the PR body itself rather than a link to an external website (ie: Jira ticket).

  • ♽ Before submitting a PR, run make test lint to avoid unnecessary CI processing. For an even more efficient workflow, consider installing pre-commit and running pre-commit install in the root of this repository.

  • ✨ We use linters to maintain clean and consistent code. Please ensure you've run make lint before submitting a PR. Some linters offer a --fix mode, which can be executed with the command make fix-linters (ensure markdownlint and golangci-lint tools are installed first).

  • 📖 If you're introducing a user-facing feature or changing existing behavior, please ensure it's properly documented.

  • 🧪 While 100% coverage isn't a requirement, we encourage unit tests for any code changes where possible.

  • 🎁 If feasible, please check if an end-to-end test can be added. See README for more details.

  • 🔎 If there's any flakiness in the CI tests, don't necessarily ignore it. It's better to address the issue before merging, or provide a valid reason to bypass it if fixing isn't possible (e.g., token rate limitations).

@chmouel
Copy link
Member

chmouel commented Dec 4, 2024

/ok-to-test

@chmouel
Copy link
Member

chmouel commented Dec 4, 2024

thanks for the contribution!

cc @zakisk

Copy link

codecov bot commented Dec 4, 2024

Codecov Report

Attention: Patch coverage is 93.61702% with 6 lines in your changes missing coverage. Please review.

Project coverage is 65.78%. Comparing base (e9a96ad) to head (4a01a2f).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
pkg/provider/provider.go 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1846      +/-   ##
==========================================
+ Coverage   65.41%   65.78%   +0.37%     
==========================================
  Files         178      178              
  Lines       13757    13839      +82     
==========================================
+ Hits         8999     9104     +105     
+ Misses       4154     4122      -32     
- Partials      604      613       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


func ValidateYaml(content []byte, filename string) error {
var validYaml any
if err := yaml.Unmarshal(content, &validYaml); err != nil {
Copy link
Contributor

@zakisk zakisk Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tricktron why did you choose "k8s.io/apimachinery/pkg/util/yaml" over "gopkg.in/yaml.v2"?, is there any specific reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tricktron why did you choose "k8s.io/apimachinery/pkg/util/yaml" over "gopkg.in/yaml.v2"?, is there any specific reason?

@zakisk Ups, I just reused the k8s.io/apimachinery/pkg/util/yaml during the refactoring from this pr. But now that you mention it, I see that k8s.io/apimachinery/pkg/util/yaml is only used once in provider/github and the gopkg.in/yaml.v2 dependency is used more often.

Therfore, it might make sense to only use gopkg.in/yaml.v2 so that we can ditch k8s.io/apimachinery/pkg/util/yaml. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zakisk I switched from k8s.io/apimachinery/pkg/util/yaml to gopkg.in/yaml.v2 for yaml validation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think if i rem ok we need gopkg.yaml for yaml error validation in that pacakge, all other we use k8 machinery yaml because it remove a lot of extra stuff when reading/writting a yaml

i think the e2e test fails on that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tricktron can you please take at this line, this is the reason E2E is failing.

@zakisk
Copy link
Contributor

zakisk commented Dec 5, 2024

/test go-testing

@chmouel
Copy link
Member

chmouel commented Dec 5, 2024

/retest

@tricktron tricktron force-pushed the f-print-bitbucket-yaml-error branch from 715af92 to 600da7e Compare December 5, 2024 10:24
@tricktron
Copy link
Contributor Author

@zakisk I don´t know why the e2e tests failed (I have not changed them). I rebased on main to get the latest commits and pushed again to run the tests.

@zakisk
Copy link
Contributor

zakisk commented Dec 5, 2024

@zakisk I don´t know why the e2e tests failed (I have not changed them). I rebased on main to get the latest commits and pushed again to run the tests.

it happens sometimes, let's wait for this new run!

@zakisk
Copy link
Contributor

zakisk commented Dec 5, 2024

@tricktron oh, I missed @chmouel 's comment.

match wrapped errors of gopkg.in/yaml.v2 instead of errors of
k8s.io/apimachinery/pkg/util/yaml.
@tricktron tricktron force-pushed the f-print-bitbucket-yaml-error branch from 2db04da to 4a01a2f Compare December 5, 2024 14:57
@tricktron
Copy link
Contributor Author

@zakisk @chmouel Alright, I also had to fix the test/testdata/TestGithubPullRequestSecondBadYaml.golden file. Now all the e2e tests pass as well.

@chmouel chmouel merged commit 78b2ea7 into openshift-pipelines:main Dec 6, 2024
5 checks passed
@chmouel
Copy link
Member

chmouel commented Dec 6, 2024

Thank you for your contribution ! @tricktron

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants