Create new `CrashReported` metric that track report upload attempts #3567

nharper285 · 2023-10-11T20:28:13Z

Summary of the Pull Request

What is this about?
We would like to know how often we attempt to upload a 'unqiue report', right now we only emit metrics when new reports are successfully uploaded

PR Checklist

Applies to work item: New CrashReported Metric Type - Represents all reports that we attempt to upload #3566
CLA signed. If not, go over here and sign the CLI.
Tests added/passed
Requires documentation to be updated
I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Info on Pull Request

What does this include?

Validation Steps Performed

How does someone test & validate?

* Remove the retention policy setting (microsoft#3452) --------- Co-authored-by: Cheick Keita <chkeita@microsoft.com>

This reverts commit c69deed.

* Redo-8.7.1-hotfix --------- Co-authored-by: Cheick Keita <chkeita@microsoft.com>

…t#3467) * Add field to ado config for checking duplicate work items * Make duplicate fields nullable and add it to python models * Update broken tests * Update docs to include new ado_duplicate_fields property

Co-authored-by: Adam <103067949+AdamL-Microsoft@users.noreply.github.com>

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.30.0 to 1.32.0. - [Release notes](https://github.com/tokio-rs/tokio/releases) - [Commits](tokio-rs/tokio@tokio-1.30.0...tokio-1.32.0) --- updated-dependencies: - dependency-name: tokio dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [elsa](https://github.com/manishearth/elsa) from 1.8.1 to 1.9.0. - [Commits](Manishearth/elsa@v1.8.1...v1.9.0) --- updated-dependencies: - dependency-name: elsa dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.7.1 to 3.8.0. - [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md) - [Commits](Stebalien/tempfile@v3.7.1...v3.8.0) --- updated-dependencies: - dependency-name: tempfile dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Updating requirements.txt to accept >= onefuzztypes. * Trying to loosen restriction.

Bumps [notify](https://github.com/notify-rs/notify) from 6.0.1 to 6.1.1. - [Release notes](https://github.com/notify-rs/notify/releases) - [Changelog](https://github.com/notify-rs/notify/blob/main/CHANGELOG.md) - [Commits](notify-rs/notify@notify-6.0.1...notify-6.1.1) --- updated-dependencies: - dependency-name: notify dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Release 8.8.0

Bumps [clap](https://github.com/clap-rs/clap) from 4.3.21 to 4.4.2. - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@v4.3.21...v4.4.2) --- updated-dependencies: - dependency-name: clap dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [gimli](https://github.com/gimli-rs/gimli) from 0.27.3 to 0.28.0. - [Changelog](https://github.com/gimli-rs/gimli/blob/master/CHANGELOG.md) - [Commits](gimli-rs/gimli@0.27.3...0.28.0) --- updated-dependencies: - dependency-name: gimli dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [clap](https://github.com/clap-rs/clap) from 4.3.21 to 4.4.2. - [Release notes](https://github.com/clap-rs/clap/releases) - [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md) - [Commits](clap-rs/clap@v4.3.21...v4.4.2) --- updated-dependencies: - dependency-name: clap dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [winreg](https://github.com/gentoo90/winreg-rs) from 0.50.0 to 0.51.0. - [Release notes](https://github.com/gentoo90/winreg-rs/releases) - [Changelog](https://github.com/gentoo90/winreg-rs/blob/master/CHANGELOG.md) - [Commits](gentoo90/winreg-rs@v0.50.0...v0.51.0) --- updated-dependencies: - dependency-name: winreg dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Adam <103067949+AdamL-Microsoft@users.noreply.github.com>

* Starting integration tests * Ready to test the test * Parametrize test * checkpoint * Test works * Run integration tests in pipeline * fmt * . * -p * Install clang * quotes not required in yaml? * Hopefully fixed windows? * Try without killondrop * lint * small test * another test * Reuse core name * Wrong step * bump tokio? * Try with rust * make build happy * Bump pete and small clean up * Clean up and make the test pass regularly * fix broken ci * Lower the poll timeout * Set the timeout in a nicer way * fix windows * fmt * Include and copy pdbs * Ignore if pdb is missing on linux * It takes too long for coverage to be generated * lint * Only warn on missing coverage since it's flaky * Fix windows build * Small clean up * Try lowering the poll delay * fix coverage * PR comments * . * Apparently make is missing? * Remove aggressive step skipping in CI

* Fix sed checks for CLI versioning * Fix. * Fix. * Changing build_cli * Trying greater than * Tring once more. * Trying major minor * trying to replace major minor * Using major minor

Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.4.0 to 1.5.0. - [Release notes](https://github.com/tokio-rs/bytes/releases) - [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md) - [Commits](tokio-rs/bytes@v1.4.0...v1.5.0) --- updated-dependencies: - dependency-name: bytes dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add more comprehensive checks and better error messages to area/iteration path validation * Join invalid chars with space instead of comma * Make tree path validation more testable * Add error code for invalid ADO project in config * Write unit tests for tree path validation * Format tree path unit tests * Merge escape character and control character checks and clarify error message

* Improve handling of unexpected breakpoints * fmt

* Fuzz coverage recording * Update cargo.toml * Update src/agent/coverage/fuzz/fuzz_targets/fuzz_target_record_coverage.rs Co-authored-by: George Pollard <porges@porg.es> * Fix fuzz --------- Co-authored-by: George Pollard <porges@porg.es>

* Reporting coverage on task start up * Moving metric up.

* Compare task version to service version * Swallow output when looking for appropriate name

* Create directories if they don't exist in the template * fmt

- [x] ability to specify a retention period on a container, which applies to newly-created blobs - [x] specify default retention periods in templates from CLI side There's a small breaking change to the Python JobHelper class.

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.7.0 to 1.8.0. - [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md) - [Commits](rayon-rs/rayon@rayon-core-v1.7.0...rayon-core-v1.8.0) --- updated-dependencies: - dependency-name: rayon dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [insta](https://github.com/mitsuhiko/insta) from 1.31.0 to 1.32.0. - [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md) - [Commits](mitsuhiko/insta@1.31.0...1.32.0) --- updated-dependencies: - dependency-name: insta dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Disable and VM CLI commands. * Formatting. * More formatting. * More formatting. * Removing Repro check.

* Make modules and coverage allowlist case insensitive on Windows * Tests and fmt * PR comments * fmt * Debugging missing file coverage * fmt * Broken linux test * Add a case insensitive transformer for better perf * cargo fix

* Tasks are selectable * Almost there * It works * fmt * remove dead code * Remove unnecessary comments * Improve instructions * fix bug * Add some dummy values for paths

…oft#3529) * Terminate process on timeout in windows for the coverage task * set the timeout before we start the debugger * split the target launch from the debugger initialization wait for the process to finish on a separate thread * fix build * move comments

…oft#3532) * Ignore regression update when the work item is in some states * format * formatting * don't hide messages in the poison queue * fix typo * update regression logic update test_template to support regression * build fix * mypy fix * build fix * move regression ignore state under ADODuplicateTemplate * replace extend with append * update set_tcp_keepalive * mke mypy happy * copy ADODuplicateTemplate.OnDuplicate.RegressionIgnoreStates

codecov-commenter · 2023-10-11T20:32:39Z

Codecov Report

Merging #3567 (f37902c) into main (46bbe5b) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #3567      +/-   ##
==========================================
- Coverage   39.05%   39.04%   -0.01%     
==========================================
  Files         302      302              
  Lines       36916    36924       +8     
==========================================
  Hits        14417    14417              
- Misses      22499    22507       +8

Files	Coverage Δ
src/agent/onefuzz-result/src/job_result.rs	`7.01% <ø> (ø)`
...gent/onefuzz-task/src/tasks/report/crash_report.rs	`10.12% <0.00%> (-0.36%)`	⬇️

src/agent/onefuzz-task/src/tasks/report/crash_report.rs

…om/nharper285/onefuzz into user/noharper/unique-report-metric

AdamL-Microsoft and others added 30 commits August 29, 2023 19:57

Release 8.7.1 (hotfix) (microsoft#3459)

c69deed

* Remove the retention policy setting (microsoft#3452) --------- Co-authored-by: Cheick Keita <chkeita@microsoft.com>

Revert "Release 8.7.1 (hotfix) (microsoft#3459)" (microsoft#3468)

c8986aa

This reverts commit c69deed.

Redo 8.7.1 (microsoft#3469)

7b40402

* Redo-8.7.1-hotfix --------- Co-authored-by: Cheick Keita <chkeita@microsoft.com>

Support custom ado fields that mark work items as duplicate (microsof…

d999603

…t#3467) * Add field to ado config for checking duplicate work items * Make duplicate fields nullable and add it to python models * Update broken tests * Update docs to include new ado_duplicate_fields property

Update readme with archive message (microsoft#3408)

b2435b1

Co-authored-by: Adam <103067949+AdamL-Microsoft@users.noreply.github.com>

Remove unnecessary method argument (microsoft#3473)

f141050

Updating requirements.txt to accept >= onefuzztypes. (microsoft#3477)

b2e6a07

* Updating requirements.txt to accept >= onefuzztypes. * Trying to loosen restriction.

Bump azure_* crates (microsoft#3478)

74475cc

Release 8.8.0 (microsoft#3466)

64699ed

* Release 8.8.0

Fix sed checks for CLI versioning (microsoft#3486)

830b479

* Fix sed checks for CLI versioning * Fix. * Fix. * Changing build_cli * Trying greater than * Tring once more. * Trying major minor * trying to replace major minor * Using major minor

Improve handling of unexpected breakpoints (microsoft#3493)

d009476

* Improve handling of unexpected breakpoints * fmt

Update azure_* crates (microsoft#3503)

18f2b4a

Fuzz coverage recording (microsoft#3322)

9ede0de

* Fuzz coverage recording * Update cargo.toml * Update src/agent/coverage/fuzz/fuzz_targets/fuzz_target_record_coverage.rs Co-authored-by: George Pollard <porges@porg.es> * Fix fuzz --------- Co-authored-by: George Pollard <porges@porg.es>

Reporting coverage on task start up (microsoft#3502)

cde6a19

* Reporting coverage on task start up * Moving metric up.

Remove feature flag from heartbeat metrics. (microsoft#3505)

1fb1563

Update archive notice. (microsoft#3507)

c7a9827

Add onefuzz service version to job created events (microsoft#3504)

58da7b4

tevoinea and others added 15 commits September 21, 2023 17:31

Tevoinea/add version checking in local tasks (microsoft#3517)

60766e6

* Compare task version to service version * Swallow output when looking for appropriate name

Create directories if they don't exist in the template (microsoft#3522)

e3c4a40

* Create directories if they don't exist in the template * fmt

Disable repro and debug VM CLI commands. (microsoft#3494)

d2ba170

* Disable and VM CLI commands. * Formatting. * More formatting. * More formatting. * Removing Repro check.

Make modules case insenstive on windows (microsoft#3527)

2c8ecc9

* Make modules and coverage allowlist case insensitive on Windows * Tests and fmt * PR comments * fmt * Debugging missing file coverage * fmt * Broken linux test * Add a case insensitive transformer for better perf * cargo fix

Update windows interceptor list (microsoft#3528)

e12b41e

Template creation command (microsoft#3531)

552df45

* Tasks are selectable * Almost there * It works * fmt * remove dead code * Remove unnecessary comments * Improve instructions * fix bug * Add some dummy values for paths

Merge branch 'microsoft:main' into main

3553220

Merge branch 'main' of https://github.com/nharper285/onefuzz

d5ffc10

Merge branch 'main' of https://github.com/nharper285/onefuzz

0bbaad8

Adding new UniqueReport metric for attempts.

285625d

Merge branch 'main' into user/noharper/unique-report-metric

864883b

nharper285 marked this pull request as ready for review October 11, 2023 20:40

nharper285 requested a review from chkeita October 11, 2023 20:40

chkeita reviewed Oct 11, 2023

View reviewed changes

src/agent/onefuzz-task/src/tasks/report/crash_report.rs Outdated Show resolved Hide resolved

nharper285 added 3 commits October 11, 2023 22:15

rename metric

2c6aaa9

lock

3ace7f3

Merge branch 'user/noharper/unique-report-metric' of https://github.c…

f37902c

…om/nharper285/onefuzz into user/noharper/unique-report-metric

nharper285 changed the title ~~Create new UniqueReport metric that track report upload attempts~~ Create new CrashReported metric that track report upload attempts Oct 11, 2023

chkeita approved these changes Oct 11, 2023

View reviewed changes

nharper285 merged commit 8c315af into microsoft:main Oct 12, 2023
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create new `CrashReported` metric that track report upload attempts #3567

Create new `CrashReported` metric that track report upload attempts #3567

nharper285 commented Oct 11, 2023

codecov-commenter commented Oct 11, 2023 •

edited

Loading

Create new CrashReported metric that track report upload attempts #3567

Create new CrashReported metric that track report upload attempts #3567

Conversation

nharper285 commented Oct 11, 2023

Summary of the Pull Request

PR Checklist

Info on Pull Request

Validation Steps Performed

codecov-commenter commented Oct 11, 2023 • edited Loading

Codecov Report

Create new `CrashReported` metric that track report upload attempts #3567

Create new `CrashReported` metric that track report upload attempts #3567

codecov-commenter commented Oct 11, 2023 •

edited

Loading