chore: add fuzzing and property based testing to Pepr #904
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
cmwylie19
requested review from
jeff-mccoy,
btlghrants and
schaeferka
as code owners
|
New and removed dependencies detected. Learn more about Socket for GitHub ↗︎
|
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
|
Current Score: 8.9 |
btlghrants
approved these changes
Jun 21, 2024
mjnagel
referenced
this pull request
in defenseunicorns/uds-core
Jul 2, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---|---|---| | [defenseunicorns/uds-common](https://togithub.com/defenseunicorns/uds-common) | | minor | `v0.6.1` -> `v0.7.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [pepr](https://togithub.com/defenseunicorns/pepr) | dependencies | patch | [`0.32.3` -> `0.32.6`](https://renovatebot.com/diffs/npm/pepr/0.32.3/0.32.6) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [registry1.dso.mil/ironbank/opensource/defenseunicorns/pepr/controller](https://togithub.com/defenseunicorns/pepr) ([source](https://repo1.dso.mil/dsop/opensource/defenseunicorns/pepr/controller)) | | patch | `v0.32.3` -> `v0.32.6` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>defenseunicorns/uds-common (defenseunicorns/uds-common)</summary> ### [`v0.7.0`](https://togithub.com/defenseunicorns/uds-common/releases/tag/v0.7.0) [Compare Source](https://togithub.com/defenseunicorns/uds-common/compare/v0.6.1...v0.7.0) ##### Features - add an image override to the renovate config ([#​160](https://togithub.com/defenseunicorns/uds-common/issues/160)) ([ea2fcf1](https://togithub.com/defenseunicorns/uds-common/commit/ea2fcf105bb7bf6a560042191ee8b7f33757c785)) - **compliance:** lula validate and evaluate capability definitions ([#​157](https://togithub.com/defenseunicorns/uds-common/issues/157)) ([b615d90](https://togithub.com/defenseunicorns/uds-common/commit/b615d90389523763d08b03f793abe5cdeeee55fc)) ##### Miscellaneous - add dockerfile `ADD` support ([#​162](https://togithub.com/defenseunicorns/uds-common/issues/162)) ([6854284](https://togithub.com/defenseunicorns/uds-common/commit/685428477abea23a2a20c311a135acc517558fc3)) - **deps:** update uds common package dependencies to v6.7.0 ([#​158](https://togithub.com/defenseunicorns/uds-common/issues/158)) ([ef9c567](https://togithub.com/defenseunicorns/uds-common/commit/ef9c567a5aea13ba2dbf22b2790b78cacb113ab1)) </details> <details> <summary>defenseunicorns/pepr (pepr)</summary> ### [`v0.32.6`](https://togithub.com/defenseunicorns/pepr/releases/tag/v0.32.6) [Compare Source](https://togithub.com/defenseunicorns/pepr/compare/v0.32.5...v0.32.6) #### What's Changed - chore: fixes bug in log message around RECONNECT events by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/931](https://togithub.com/defenseunicorns/pepr/pull/931) - chore: bump github/codeql-action from 3.25.10 to 3.25.11 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/928](https://togithub.com/defenseunicorns/pepr/pull/928) - chore: bump chainguard/node-lts from `b12b298` to `437a945` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/929](https://togithub.com/defenseunicorns/pepr/pull/929) - chore: bump kubernetes-fluent-client from 2.6.2 to 2.6.3 in the production-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/932](https://togithub.com/defenseunicorns/pepr/pull/932) **Release was prefaced by a 15 hour soak on the informer** **Full Changelog**: defenseunicorns/pepr@v0.32.5...v0.32.6 ### [`v0.32.5`](https://togithub.com/defenseunicorns/pepr/releases/tag/v0.32.5) [Compare Source](https://togithub.com/defenseunicorns/pepr/compare/v0.32.4...v0.32.5) #### What's Changed - chore: document sdk by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/922](https://togithub.com/defenseunicorns/pepr/pull/922) - chore: bump prom-client from 15.1.2 to 15.1.3 in the production-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/923](https://togithub.com/defenseunicorns/pepr/pull/923) **Full Changelog**: defenseunicorns/pepr@v0.32.4...v0.32.5 ### [`v0.32.4`](https://togithub.com/defenseunicorns/pepr/releases/tag/v0.32.4) [Compare Source](https://togithub.com/defenseunicorns/pepr/compare/v0.32.3...v0.32.4) #### CLI Updates - chore: deploy image pull secret by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/911](https://togithub.com/defenseunicorns/pepr/pull/911) ```bash --pullSecret <name>, "Deploy imagePullSecret for Controller private registry" --docker-server <server>, "Docker server address" --docker-username <username>, "Docker registry username" --docker-email <email>, "Email for Docker registry" --docker-password <password>, "Password for Docker registry" ``` - chore: optional pull secret by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/908](https://togithub.com/defenseunicorns/pepr/pull/908) ```bash --withPullSecret <imagePullSecret> ``` #### What's Changed - chore: add fuzzing and property based testing to Pepr by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/904](https://togithub.com/defenseunicorns/pepr/pull/904) - chore: fix unparsable lines in monitor command by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/907](https://togithub.com/defenseunicorns/pepr/pull/907) - chore: document sdk by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/919](https://togithub.com/defenseunicorns/pepr/pull/919) - chore: bump chainguard/node-lts from `5bcab86` to `4214789` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/903](https://togithub.com/defenseunicorns/pepr/pull/903) - chore: bump [@​types/node](https://togithub.com/types/node) from 18.19.36 to 18.19.37 in the development-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/905](https://togithub.com/defenseunicorns/pepr/pull/905) - chore: bump chainguard/node-lts from `4214789` to `e68c211` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/906](https://togithub.com/defenseunicorns/pepr/pull/906) - chore: bump chainguard/node-lts from `e68c211` to `a07cbcb` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/909](https://togithub.com/defenseunicorns/pepr/pull/909) - chore: bump the development-dependencies group with 2 updates by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/910](https://togithub.com/defenseunicorns/pepr/pull/910) - chore: bump chainguard/node-lts from `a07cbcb` to `d38ff06` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/913](https://togithub.com/defenseunicorns/pepr/pull/913) - chore: bump [@​types/node](https://togithub.com/types/node) from 18.19.38 to 18.19.39 in the development-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/912](https://togithub.com/defenseunicorns/pepr/pull/912) - chore: bump chainguard/node-lts from `d38ff06` to `b12b298` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/917](https://togithub.com/defenseunicorns/pepr/pull/917) **Full Changelog**: defenseunicorns/pepr@v0.32.3...v0.32.4 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/defenseunicorns/uds-core). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjAuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyMS45IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Micah Nagel <micah.nagel@defenseunicorns.com>
](https://renovatebot.com){kind=link}
rjferguson21
referenced
this pull request
in defenseunicorns/uds-core
Jul 11, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---|---|---| | [defenseunicorns/uds-common](https://togithub.com/defenseunicorns/uds-common) | | minor | `v0.6.1` -> `v0.7.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [pepr](https://togithub.com/defenseunicorns/pepr) | dependencies | patch | [`0.32.3` -> `0.32.6`](https://renovatebot.com/diffs/npm/pepr/0.32.3/0.32.6) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [registry1.dso.mil/ironbank/opensource/defenseunicorns/pepr/controller](https://togithub.com/defenseunicorns/pepr) ([source](https://repo1.dso.mil/dsop/opensource/defenseunicorns/pepr/controller)) | | patch | `v0.32.3` -> `v0.32.6` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>defenseunicorns/uds-common (defenseunicorns/uds-common)</summary> ### [`v0.7.0`](https://togithub.com/defenseunicorns/uds-common/releases/tag/v0.7.0) [Compare Source](https://togithub.com/defenseunicorns/uds-common/compare/v0.6.1...v0.7.0) ##### Features - add an image override to the renovate config ([#​160](https://togithub.com/defenseunicorns/uds-common/issues/160)) ([ea2fcf1](https://togithub.com/defenseunicorns/uds-common/commit/ea2fcf105bb7bf6a560042191ee8b7f33757c785)) - **compliance:** lula validate and evaluate capability definitions ([#​157](https://togithub.com/defenseunicorns/uds-common/issues/157)) ([b615d90](https://togithub.com/defenseunicorns/uds-common/commit/b615d90389523763d08b03f793abe5cdeeee55fc)) ##### Miscellaneous - add dockerfile `ADD` support ([#​162](https://togithub.com/defenseunicorns/uds-common/issues/162)) ([6854284](https://togithub.com/defenseunicorns/uds-common/commit/685428477abea23a2a20c311a135acc517558fc3)) - **deps:** update uds common package dependencies to v6.7.0 ([#​158](https://togithub.com/defenseunicorns/uds-common/issues/158)) ([ef9c567](https://togithub.com/defenseunicorns/uds-common/commit/ef9c567a5aea13ba2dbf22b2790b78cacb113ab1)) </details> <details> <summary>defenseunicorns/pepr (pepr)</summary> ### [`v0.32.6`](https://togithub.com/defenseunicorns/pepr/releases/tag/v0.32.6) [Compare Source](https://togithub.com/defenseunicorns/pepr/compare/v0.32.5...v0.32.6) #### What's Changed - chore: fixes bug in log message around RECONNECT events by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/931](https://togithub.com/defenseunicorns/pepr/pull/931) - chore: bump github/codeql-action from 3.25.10 to 3.25.11 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/928](https://togithub.com/defenseunicorns/pepr/pull/928) - chore: bump chainguard/node-lts from `b12b298` to `437a945` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/929](https://togithub.com/defenseunicorns/pepr/pull/929) - chore: bump kubernetes-fluent-client from 2.6.2 to 2.6.3 in the production-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/932](https://togithub.com/defenseunicorns/pepr/pull/932) **Release was prefaced by a 15 hour soak on the informer** **Full Changelog**: defenseunicorns/pepr@v0.32.5...v0.32.6 ### [`v0.32.5`](https://togithub.com/defenseunicorns/pepr/releases/tag/v0.32.5) [Compare Source](https://togithub.com/defenseunicorns/pepr/compare/v0.32.4...v0.32.5) #### What's Changed - chore: document sdk by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/922](https://togithub.com/defenseunicorns/pepr/pull/922) - chore: bump prom-client from 15.1.2 to 15.1.3 in the production-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/923](https://togithub.com/defenseunicorns/pepr/pull/923) **Full Changelog**: defenseunicorns/pepr@v0.32.4...v0.32.5 ### [`v0.32.4`](https://togithub.com/defenseunicorns/pepr/releases/tag/v0.32.4) [Compare Source](https://togithub.com/defenseunicorns/pepr/compare/v0.32.3...v0.32.4) #### CLI Updates - chore: deploy image pull secret by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/911](https://togithub.com/defenseunicorns/pepr/pull/911) ```bash --pullSecret <name>, "Deploy imagePullSecret for Controller private registry" --docker-server <server>, "Docker server address" --docker-username <username>, "Docker registry username" --docker-email <email>, "Email for Docker registry" --docker-password <password>, "Password for Docker registry" ``` - chore: optional pull secret by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/908](https://togithub.com/defenseunicorns/pepr/pull/908) ```bash --withPullSecret <imagePullSecret> ``` #### What's Changed - chore: add fuzzing and property based testing to Pepr by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/904](https://togithub.com/defenseunicorns/pepr/pull/904) - chore: fix unparsable lines in monitor command by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/907](https://togithub.com/defenseunicorns/pepr/pull/907) - chore: document sdk by [@​cmwylie19](https://togithub.com/cmwylie19) in [https://github.com/defenseunicorns/pepr/pull/919](https://togithub.com/defenseunicorns/pepr/pull/919) - chore: bump chainguard/node-lts from `5bcab86` to `4214789` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/903](https://togithub.com/defenseunicorns/pepr/pull/903) - chore: bump [@​types/node](https://togithub.com/types/node) from 18.19.36 to 18.19.37 in the development-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/905](https://togithub.com/defenseunicorns/pepr/pull/905) - chore: bump chainguard/node-lts from `4214789` to `e68c211` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/906](https://togithub.com/defenseunicorns/pepr/pull/906) - chore: bump chainguard/node-lts from `e68c211` to `a07cbcb` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/909](https://togithub.com/defenseunicorns/pepr/pull/909) - chore: bump the development-dependencies group with 2 updates by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/910](https://togithub.com/defenseunicorns/pepr/pull/910) - chore: bump chainguard/node-lts from `a07cbcb` to `d38ff06` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/913](https://togithub.com/defenseunicorns/pepr/pull/913) - chore: bump [@​types/node](https://togithub.com/types/node) from 18.19.38 to 18.19.39 in the development-dependencies group by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/912](https://togithub.com/defenseunicorns/pepr/pull/912) - chore: bump chainguard/node-lts from `d38ff06` to `b12b298` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/defenseunicorns/pepr/pull/917](https://togithub.com/defenseunicorns/pepr/pull/917) **Full Changelog**: defenseunicorns/pepr@v0.32.3...v0.32.4 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/defenseunicorns/uds-core). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjAuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyMS45IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Micah Nagel <micah.nagel@defenseunicorns.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Today in Pepr testing, we specify the inputs and outputs we want to use and observe.
The problem with our current approach to testing software is that it places the responsibility of identifying bugs solely on the developers writing the tests. They must think about all the things that could go wrong if they want their tests to find all potential bugs. While it's possible to think about common failure cases with enough experience, it's not really feasible to consider all possible edge cases without proving the code line by line.
This PR intends to start experimentally adding property based, fuzzing and fake data tests to Pepr.
We think it should also increase our Scorecard score
Shows library injecting random strings
Related Issue
Fixes #742
Relates to #
Type of change
Checklist before merging