Unable to Detect More Than 900 Objects Using CountGD on Hugging Face

[chickencoins]

Unable to Detect More Than 900 Objects Using CountGD on Hugging Face

Description

According to the CountGD openreview and the discussion provided, CountGD is designed to handle images containing more than 900 objects by utilizing an adaptive cropping scheme detailed in Appendix D. This approach is intended to overcome the limitations of previous methods that use a fixed number of object queries (e.g., GroundingDINO), which cap the maximum number of predicted objects at 900.

However, when using the CountGD implementation on Hugging Face, I encountered an issue where submitting images with more than 900 objects resulted in only 900 objects being detected. This behavior suggests that the current Hugging Face implementation may not support processing images with more than 900 objects as expected.

Current Behavior

When submitting images with more than 900 objects to the CountGD model on Hugging Face, only 900 objects are detected, indicating that the adaptive cropping mechanism may not be functioning as intended in this implementation.

Question

How can I enable or utilize the adaptive cropping scheme in the Hugging Face implementation of CountGD to accurately detect and count more than 900 objects in an image? Are there specific configurations, parameters, or steps that need to be followed to activate this functionality?

I'm currently using a docker pull on the huggings face because of the environmental adjustment issue. Of course, the same problem happened in this case.
I really wish I could fix this issue by modifying some of the codes inside the docker container I pulled.
Any guidance or suggestions to resolve this issue would be greatly appreciated. Thank you!

[niki-amini-naieni]

Hi, thank you for your question and interest! The Hugging Face app does not currently support this capability. However, the CountGD repository (from which the app was built) does. Please see this closed issue regarding this. The relevant code is here. Intuitively, the adaptive cropping just divides the original image into smaller pieces and counts the objects in each smaller piece, while averaging overlapping regions. You could just do this yourself. For example, below I have found a square with missing counts in your image (outlined in red):

I took a screenshot of this crop and passed it to CountGD with the same prompt as you. As you can see (circled in yellow), CountGD counts some of the missing plants in that region now.

You just need to divide up the image into these pieces with some overlapping regions whose counts are averaged. As for the size of the crop window, I used the provided exemplars. I see in your case you do not provide exemplars. The idea is if you have an approximate pixel size for each object (e.g., 30 x 30 pixels), you can determine a crop window with an upper limit on the number of objects. This will ensure that far fewer than 900 objects appear in each crop window. You can also use the exemplar to determine the width of the overlapping region to ensure each object appears fully in at least one crop.