Using this resource as a python module, contributing to creating a .config file

[bw4sz]

Hello @Vincentqyw, thanks for your work on this tool. I am a researcher at the University Florida and an open-source developer for machine learning models for biological monitoring (https://deepforest.readthedocs.io/en/latest/). Our users often have overlapping non-georeferenced images in which there are duplicate objects and we want a unique count.

Example image with birds

Screenshot of a pair of matched image from the demo. There are birds in each image that overlap taken from a plane as it passes an area.

Our goal is to not to create a stitched image, but rather identify the overlapping detections among images to create a unique set.

I can imagine writing a workflow in our python module to

1. Run our object detection modules to predict the location and classes of objects in the image. In this case, birds.
2. Call your local GUI or pipe images to the docker container and get results.
3. Remove duplicate detections in the warped images and return users the raw images with only the unique non-overlapping detections among a set of images.

This is all hypothetical, but my general sense is that there is alot of desire from the automated airborne/underwater biological monitoring community, which I help organize.

Questions

1. Is this repo the right choice for this project? Can I interact with the module through python or command line?
2. What do you see as the challenges here?

Thanks for your time.

[Vincentqyw]

Thank you for your interest in our project. I have a clear understanding of the context of your project. Your goal is to perform overlapping image checks, and indeed, the method of image local feature matching can meet your requirements. For instance, you can match features of images A and B, calculate the homography matrix H, and then warp image A onto image B to find the overlapping area.

The project, image-matching-webui, is built based on hloc. It collects numerous modern local features and matchers. The main function of the webui is to provide a user interface and a general interface, allowing users to quickly select the features or matchers they need.

I believe that using this webui entirely might be too heavy-duty for your project. You could try to find a suitable matcher for your task based on the webui, then locate it in the third_party and attempt to integrate it into your project.

The challenges I foresee are as follows:

• Forest scenes in your example belong to similar texture scenes, where current feature matchers might struggle to perform well. Even one of the most advanced algorithms today, Roma, requires more computing resources and time.
• The estimation of overlap assumes a plane. If your photos are taken close to the ground, this assumption might be problematic. If they are taken from a distance, this assumption should be satisfactory. However, it might pose difficulties for your object detection module.

Hope this response provides some guidance for your project.

[bw4sz]

Perfect, thanks.