Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix detection prediction outputs #3634

Merged

Conversation

sungchul2
Copy link
Contributor

@sungchul2 sungchul2 commented Jun 18, 2024

Summary

Ticket no. : 144396

Torch models have abnormal outputs, which seem unfiltered (especially ATSS and SSD).
It can yield wrong performance measurements by adding very many false positives.

box color : blue - GT | red - predictions
image

But, OV model have normal outputs because outputs are filtered before wrapped by DetectionResult

figure order : rtmdet_tiny OV Model | rtmdet_tiny Torch Model | atss_mobilenetv2 Torch Model
image

To fix this issue, this PR includes:

  • Use confidence_threshold to filter outputs
    • model api uses it (default : 0.5) to filter outputs (link). To match the results, use confidence_threshold from checkpoint before wrapping outputs with DetBatchPredEntity.
      figure order : rtmdet_tiny OV Model | rtmdet_tiny Torch Model
      image

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have ran e2e tests and there is no issues.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

eunwoosh
eunwoosh previously approved these changes Jun 20, 2024
chuneuny-emily
chuneuny-emily previously approved these changes Jun 20, 2024
@sungchul2 sungchul2 dismissed stale reviews from chuneuny-emily and eunwoosh via 5ede797 June 20, 2024 05:49
@sungchul2 sungchul2 enabled auto-merge (squash) June 20, 2024 07:27
@sungchul2 sungchul2 merged commit 4280d7b into openvinotoolkit:develop Jun 20, 2024
13 checks passed
@sungchul2 sungchul2 deleted the fix-detection-pred-outputs branch June 20, 2024 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants