Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Continuous Fuzzing Integration via fuzzit.dev #1001

Merged
merged 1 commit into from
Aug 27, 2019

Conversation

yevgenypats
Copy link

This PR adds a continuous fuzzing integration to grpc-gateway Circle pipeline via Fuzzit service.

This means the following:

  • Every time new code is pushed to master new fuzzers are built and push to fuzzit where they run continuously generate new test-cases and alerts if new crashes are found
  • Every pull-request the fuzzers runs through a quick regression tests with the generated test-cases from the previous step. If something new/old bugs are introduced you will see this immediately in the Travis check.

To take ownership of the organisation, please login to https://app.fuzzit.dev and let me know your account.

This is PR relates to this issue:
#998

Also this PR lays the foundation with the first basic go-fuzz target function that fuzzes the http-rule.

Please review and feel free to comment/ask questions

cc @johanbrandhorst

@googlebot
Copy link

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@yevgenypats
Copy link
Author

@dominikh can you confirm with @googlebot I consent

@codecov-io
Copy link

codecov-io commented Aug 20, 2019

Codecov Report

Merging #1001 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1001   +/-   ##
=======================================
  Coverage   53.45%   53.45%           
=======================================
  Files          40       40           
  Lines        4028     4028           
=======================================
  Hits         2153     2153           
  Misses       1676     1676           
  Partials      199      199

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fd2d159...4bce0b1. Read the comment docs.

@johanbrandhorst
Copy link
Collaborator

I think @achew22 is better suited for the org ownership role. I will take a look at the code.

Copy link
Collaborator

@johanbrandhorst johanbrandhorst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just one small question.

@@ -48,6 +48,14 @@ jobs:
- checkout
- run: go get golang.org/x/lint/golint
- run: make lint
fuzzit:
docker:
- image: fuzzitdev/golang:1.12.7-buster
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be any value in using something like the latest tag instead of this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the Dockerfile is here

the tag is the version of golang and it's always the latest (we push to that tag if there are changes) though it doesn't really change often as it just golang+clang docker

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be nice to have it automatically update, could we make it latest?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean by latest? latest golang or latest golang:1.12.7

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fuzzitdev/golang:latest. Is this not a tag you will support?

@dominikh
Copy link
Contributor

@googlebot I consent

@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@johanbrandhorst
Copy link
Collaborator

I'm confused by the build error, does it mean anything to you?

@yevgenypats
Copy link
Author

@johanbrandhorst first time I'm seeing this. very strange.

@yevgenypats
Copy link
Author

I'll try to solve this tomorrow morning

fuzzit.sh Outdated Show resolved Hide resolved
fuzzit.sh Show resolved Hide resolved
@yevgenypats yevgenypats force-pushed the fuzzit branch 4 times, most recently from 5acff46 to 53f0222 Compare August 21, 2019 16:47
Add a basic fuzz test for the httprule package, run regression tests
locally on every PR and continuous fuzzing on fuzzit.dev.
@yevgenypats
Copy link
Author

@johanbrandhorst Looks like it works! you need to sign up at https://app.fuzzit.dev so I can add you to grpc-gateway account so you can take over (let me know once you sign-up). You will need to copy the key from https://app.fuzzit.dev/orgs/grpc-gateway/settings to CircleCI environment variable FUZZIT_API_KEY

@johanbrandhorst
Copy link
Collaborator

Awesome, great job, as I mentioned I nominate @achew22 to be our org manager, he can make the CircleCI changes too. Andrew could you please merge this once that's done?

@achew22
Copy link
Collaborator

achew22 commented Aug 22, 2019

The link of https://app.fuzzit.dev/orgs/grpc-gateway/settings doesn't seem to exist. I'm guessing there is a manual step on @yevgenypats's part to create that. I have created an account through the github oauth login, the email on file is the same one I commit to this repo under. I presume that the API key for fuzzit isn't secret, can you confirm that for me?

@yevgenypats
Copy link
Author

@achew22 you should have access now. The api key is indeed a secret and this why it should be stored in the CircleCI environment variable.

@yevgenypats
Copy link
Author

@achew22 ping

@yevgenypats
Copy link
Author

@johanbrandhorst @achew22 friendly ping:)

@johanbrandhorst
Copy link
Collaborator

johanbrandhorst commented Aug 27, 2019

@yevgenypats I've registered with fuzzit, could you add me as an admin to the grpc-gateway?

@yevgenypats
Copy link
Author

@johanbrandhorst done. you should have access now to https://app.fuzzit.dev/orgs/grpc-gateway/dashboard

@johanbrandhorst
Copy link
Collaborator

I've added the FUZZIT_API_KEY to our circle env vars. Is this good to merge?

@yevgenypats
Copy link
Author

Yes!

@johanbrandhorst johanbrandhorst merged commit 9d0c596 into grpc-ecosystem:master Aug 27, 2019
@johanbrandhorst
Copy link
Collaborator

Thanks for getting this in @yevgenypats :)

@yevgenypats
Copy link
Author

Sure! feel free to RT:) https://twitter.com/fuzzitdev/status/1166282998125334528

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants