Skip to content
This repository has been archived by the owner on Nov 1, 2023. It is now read-only.

add end-to-end integration testing of fuzzing pipelines #46

Merged
merged 5 commits into from
Sep 29, 2020
Merged

add end-to-end integration testing of fuzzing pipelines #46

merged 5 commits into from
Sep 29, 2020

Conversation

bmc-msft
Copy link
Contributor

@bmc-msft bmc-msft commented Sep 29, 2020 •
edited
Loading

Summary of the Pull Request

Migrates the majority of internal CICD tooling for integration testing into something consumable outside of Azure Devops.

PR Checklist

  • CLA signed. If not, go over here and sign the CLI.
  • Tests added/passed
  • I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Info on Pull Request

  • Each of the job templates now returns the Job (in the case of --dryrun, returns None)
  • Uses the onefuzz-samples released artifacts of basic fuzzing binaries for testing.

The process followed:

  1. Creates pools & scalesets needed (optionally uses existing pools if the user specifies their pools on the cmdlien)
  2. Launches the templates specifying the above pools
  3. Monitors each of the launched jobs until either the tasks fail or a report is created in the job specific container
  4. Launches repro VMs for all of the successful tasks using one of the reports report
  5. Monitors each of the repro VMs until either it fails or verifies the rip register works as expected.
  6. Cleans up all created resources

This assumes you have the platform specific debugger (cdb for windows, gdb for linux) in your path. If you're launching this script within WSL, this is possible. Otherwise, use --skip_repro or --os_list to limit to the OS.

Validation Steps Performed

cd src/cli; examples/integration-test.py test ~/onefuzz-samples/

@bmc-msft bmc-msft merged commit aa9332b into microsoft:main Sep 29, 2020
anshuman-goel pushed a commit to anshuman-goel/onefuzz that referenced this pull request Oct 1, 2020
@bmc-msft @anshuman-goel
add end-to-end integration testing of fuzzing pipelines (microsoft#46)
25a43ce
anshuman-goel added a commit to anshuman-goel/onefuzz that referenced this pull request Oct 6, 2020
@anshuman-goel @ranweiler
@bmc-msft @demoray @chkeita
User/ansgoel/pool scale 1 (#1)
f33dc90 
* newer onefuzz type

* update to types

* scaling

* optional region

* region as optional

* pool as mandatory

* cap case

* newer changes

* idk

* af errors

* pool debugging

* region cap case

* task state

* Update `can_schedule` check to support node reimaging (microsoft#35)

- Include version in `can_schedule` check, expect rich response
- Check if work can be scheduled before claiming
- If work is stopped, claim and drop it
- If node is outdated, don't claim work

* Refactor internal node event schemas (microsoft#29)

* use sccache more consistently  (microsoft#47)

- Uses actions/cache via `src/ci/rust-prereqs.sh` to cache installing multiple rust prereqs including `cargo-audit` and `sccache`
- Uses actions/cache to cache `src/agent/target` and `src/proxy-manager/target` 

Co-authored-by: Brian Caswell <bmc@shmoo.com>

* add end-to-end integration testing of fuzzing pipelines (microsoft#46)

* re-add black to lint stages (microsoft#45)

* fix formatting (microsoft#55)

* Example sdk in azure functions (microsoft#56)

* Adding node assignment to the task entity (microsoft#54)

* Link VMSS nodes and tasks when setting up (microsoft#43)

Add a (backwards-compatible) data field to node state update events. Use this to link nodes and tasks as soon as the tasks have been claimed. Add a new task-level `setting_up` state to encode this.

* set more detailed version information during builds (microsoft#58)

* Using a clean flag (microsoft#59)

Locally adding `--clean` to the command helps in making a exe w/o caching.

* Remove use of `batch` in NodeMessages (microsoft#60)

* make version.localchanges match API logic (microsoft#62)

* Set log levels in Azure Functions by hand for 3rd party libraries (microsoft#63)

* use sc.exe instead of Set-Content (microsoft#67)

* move to warning (microsoft#66)

* slim down msg (microsoft#65)

* only set stating to stopping (microsoft#64)

* setting a default

* reversing things

* agent comment

* debugging

* scaling down

* fixing resizes

* Build 1.1.0 (microsoft#99)

* Update CURRENT_VERSION (microsoft#104)

* newer changes

* api fix

* removing onefuzztypes

* linter errors

* syntax error

* sorting imports

* linter

* linter

* linting fixes

* linting fixes

* import sort

* linting fixes

* linting fixes

* fixes

* removing non needed changes

* model remove non needed changes

Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>
Co-authored-by: bmc-msft <41130664+bmc-msft@users.noreply.github.com>
Co-authored-by: Brian Caswell <bmc@shmoo.com>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
@ghost ghost locked as resolved and limited conversation to collaborators Apr 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@chkeita chkeita chkeita approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bmc-msft @chkeita @demoray