Identifying individual members of animal species (multiple species and individuals ideally)

[tyeth]

Hey, long story short, I want to track individual members of an animal population (nuisance animals [pidgeons, gulls, cats] and interesting but not to be over-fed animals [squirrels/foxes/badgers/all other medium-large birds], and unlimited feeding for small birds). Most of them will be seen again, (therefore potentially before), but not all. It's a Washing line in the garden with 4 bird feeders. The camera will be mounted in the greenhouse looking out at them, so catches all sorts in the garden (and will be quite a bad background to work with for identifying small birds). The idea is two fold, one to have nature photos that are not just motion trigger, and secondly (most importantly) stop overfeeding certain greedy individuals, so I need to identify the animal type and then segment into known existing individuals with confidence ratings (and i guess a matching unknown individual upon which a profile is being built).

This needs to be cutting edge like the video I watched last year (and now cant find) about tracking disease / genetic traits in the back legs of big cats through individual and gait analysis, and action/emotion + individual analysis in primates.

Any pointers in this or these directions would be great

[capjamesg]

Hello @tyeth! Thank you for opening this discussion.

It sounds like you are interested in ReID, a concept that refers to re-identifying objects that have already appeared in view before. This is not a topic about which I know a lot, but I thought I would mention it as a direction for your research. With that said, ReID is a complex technical problem without a single defined solution at the moment.

You could train a vision model that identifies certain animals like gulls and cats for general detection, though. Your message inspired be to integrate BioCLIP into Autodistill. You can use BioCLIP with Autodistill to classify images of animals. You can then use your classified images as input to train a model. You can also combine BioCLIP with a detection model like Grounding DINO to:

1. Detect all birds / cats, etc. with Grounding DINO, a model that works well at identifying generic objects.
2. Classify the species of each animal with BioCLIP, a model that knows a lot about animals and plants.

BioCLIP was trained on a massive animal and plant dataset, so there may be some experimentation you can do there with regard to specific characteristics of an animal (i.e. fur colour, etc.).

You can use the resulting dataset that was labeled with Autodistill, Grounding DINO, and BioCLIP to train a fine-tuned model which will run in close-to-real-time. You can deploy that model with Inference.

Let me know if you have any questions!