Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.github/workflows: Enable fuzz testing as a nightly CI task #631

Merged
merged 1 commit into from
Sep 27, 2023
Merged

.github/workflows: Enable fuzz testing as a nightly CI task #631

merged 1 commit into from
Sep 27, 2023

Conversation

benjamindoron
Copy link
Contributor

@benjamindoron benjamindoron commented Aug 11, 2023 •
edited
Loading

TODO:

  • We should finalise max length to fuzz per target.
  • Parameterise runs-on to consider using GCP?

@benjamindoron benjamindoron force-pushed the fuzzing_ci branch 2 times, most recently from 687818b to 29b5870 Compare August 16, 2023 14:56
korran
korran previously approved these changes Aug 16, 2023
View reviewed changes
@benjamindoron benjamindoron force-pushed the fuzzing_ci branch 2 times, most recently from e51cb5a to 18bb259 Compare August 24, 2023 18:34
@benjamindoron benjamindoron mentioned this pull request Aug 26, 2023
Merged
@benjamindoron benjamindoron force-pushed the fuzzing_ci branch 2 times, most recently from 6da7119 to 2b3657f Compare August 31, 2023 19:27
jhand2
jhand2 previously approved these changes Sep 1, 2023
View reviewed changes
@FerralCoder FerralCoder added the Build & Release Build and release process, CI automation label Sep 11, 2023
@benjamindoron benjamindoron changed the title [DNM] .github/workflows: Enable fuzz testing as a nightly CI task .github/workflows: Enable fuzz testing as a nightly CI task Sep 18, 2023
@benjamindoron benjamindoron force-pushed the fuzzing_ci branch 3 times, most recently from b691c7c to 4fcce47 Compare September 22, 2023 18:51
@jhand2 jhand2 enabled auto-merge (rebase) September 27, 2023 16:34
@benjamindoron @jhand2
[DNM] .github/workflows: Enable fuzz testing as a nightly CI task
3c4ac82 
We should finalise max length to fuzz per target.
Parameterise `runs-on` to consider using GCP?

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
@jhand2 jhand2 merged commit 2a4a37f into chipsalliance:main Sep 27, 2023
@benjamindoron
Copy link
Contributor Author

Sure, I think the max length values are at least good start now.

@benjamindoron
Copy link
Contributor Author

benjamindoron commented Sep 28, 2023 •
edited
Loading

Necessary follow-ups:

  • libFuzzer: GH ignore return value (target crash is not CI error). max_total_time does not work as expected, jobs continue to be spawned. image/verify corpus unavailable, though the job is dependent, it ran on another system.
  • AFL: Fixup shell script syntax error. Consider disabling for now, or look into exit codes to identify error.

Then, switch to GCP.

benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Oct 3, 2023
@benjamindoron
[DNM] .github/workflows: Fix fuzz testing CI
aaecb5c 
Addresses
chipsalliance#631 (comment).

Now runs each worker for a limited amount of time, totalling one hour.
AFL "run fuzzing" now passes ShellCheck.

DNM: Investigate AFL exit codes or disable for now.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
@benjamindoron benjamindoron mentioned this pull request Oct 5, 2023
Open
jhand2 pushed a commit to benjamindoron/caliptra-sw that referenced this pull request Oct 31, 2023
@benjamindoron @jhand2
[DNM] .github/workflows: Fix fuzz testing CI
e2851ca 
Addresses
chipsalliance#631 (comment).

Now runs each worker for a limited amount of time, totalling one hour.
AFL "run fuzzing" now passes ShellCheck.

DNM: Investigate AFL exit codes or disable for now.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
jhand2 pushed a commit to benjamindoron/caliptra-sw that referenced this pull request Nov 9, 2023
@benjamindoron @jhand2
[DNM] .github/workflows: Fix fuzz testing CI
023a3af 
Addresses
chipsalliance#631 (comment).

Now runs each worker for a limited amount of time, totalling one hour.
AFL "run fuzzing" now passes ShellCheck.

DNM: Investigate AFL exit codes or disable for now.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Nov 9, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
ebe2cd7 
Addresses chipsalliance#631 (comment).

Now runs each worker for a limited amount of time, totalling one hour.
AFL "run fuzzing" now passes ShellCheck.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Nov 10, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
0092ea3 
Addresses chipsalliance#631 (comment).

Now runs each worker for a limited amount of time, totalling one hour.
AFL "run fuzzing" now passes ShellCheck.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
jhand2 pushed a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 6, 2023
@benjamindoron @jhand2
.github/workflows: Fix fuzz testing CI
21f5b07 
Addresses chipsalliance#631 (comment).

Now runs each worker for a limited amount of time, totalling one hour.
AFL "run fuzzing" now passes ShellCheck.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 16, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
b111a3c 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 16, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
b3af958 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 16, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
d618e27 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 16, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
9a84294 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 16, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
db8827c 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 18, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
28df906 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Dec 18, 2023
@benjamindoron
.github/workflows: Fix fuzz testing CI
834930c 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Jun 27, 2024
@benjamindoron
.github/workflows: Fix fuzz testing CI
b9e6f5f 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Jun 27, 2024
@benjamindoron
.github/workflows: Fix fuzz testing CI
87a8fc2 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Jun 27, 2024
@benjamindoron
.github/workflows: Fix fuzz testing CI
5e7625c 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Jun 27, 2024
@benjamindoron
.github/workflows: Fix fuzz testing CI
8917b9e 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Jul 4, 2024
@benjamindoron
.github/workflows: Fix fuzz testing CI
f3d328d 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
benjamindoron added a commit to benjamindoron/caliptra-sw that referenced this pull request Aug 12, 2024
@benjamindoron
.github/workflows: Fix fuzz testing CI
47fc120 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
leongross pushed a commit to benjamindoron/caliptra-sw that referenced this pull request Feb 5, 2025
@benjamindoron @leongross
.github/workflows: Fix fuzz testing CI
250375e 
Addresses chipsalliance#631 (comment).

Attempts to use the "workers" argument to gain parallelisation and
better statefulness.
AFL "run fuzzing" now passes ShellCheck, but seems to run overtime.

Signed-off-by: Benjamin Doron <benjamin.doron@9elements.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jhand2 jhand2 jhand2 approved these changes

@korran korran korran left review comments

Assignees
No one assigned
Labels
Build & Release Build and release process, CI automation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@benjamindoron @korran @jhand2 @FerralCoder