Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix F1 auto-threshold to choose best largest confidence #2367

Closed

Conversation

goodsong81
Copy link

@goodsong81 goodsong81 commented Jul 17, 2023

Summary

  • Current adaptive threshold logic chooses the best "smallest" confidence, which might result in too small threshold, thus high level of false alarm ratio.
  • Result-based threshold: before: 0.025 -> after fix: 0.425
    image

How to test

  • Unit test adjusted for this edge case

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

Signed-off-by: Songki Choi <songki.choi@intel.com>
@goodsong81 goodsong81 added the FIX defect fix label Jul 17, 2023
@goodsong81 goodsong81 added this to the 1.4.0 milestone Jul 17, 2023
@github-actions github-actions bot added API Any changes in OTX API TEST Any changes in tests labels Jul 17, 2023
Signed-off-by: Songki Choi <songki.choi@intel.com>
@goodsong81 goodsong81 marked this pull request as ready for review July 17, 2023 13:25
@goodsong81 goodsong81 requested a review from a team as a code owner July 17, 2023 13:25
@goodsong81
Copy link
Author

@sungmanc Could you check if this change could affect the small + low-positive scenario?

@sungmanc
Copy link
Contributor

@sungmanc Could you check if this change could affect the small + low-positive scenario?

Sure, I'll check with some datasets

jaegukhyun
jaegukhyun previously approved these changes Jul 18, 2023
@github-actions github-actions bot added the DOC Improvements or additions to documentation label Jul 18, 2023
Songki Choi added 2 commits July 18, 2023 13:35
Signed-off-by: Songki Choi <songki.choi@intel.com>
@goodsong81
Copy link
Author

Retarget to develop: #2371

@goodsong81 goodsong81 closed this Jul 18, 2023
@goodsong81 goodsong81 deleted the fix/f1-auto-thresh branch July 18, 2023 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Any changes in OTX API DOC Improvements or additions to documentation FIX defect fix TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants