Identifying Near Duplicate videos | A real world example

You are wasting time on Reddit and you came across the following meme, and now you would like to make your own meme using the original video but how do you find the original video? You can actually ask the OP but let's assume they wouldn't tell you.

text.mp4

_{Video: https://user-images.githubusercontent.com/64683866/148959826-9b83c1ad-f9ef-4054-83df-d420ca318de6.mp4}

_{Source: https://www.reddit.com/r/IndianDankMemes/comments/s0bikx/yes_guys/}

You want to get to the original video, the one just below this line. But how do you make it happen? Unlike images, we don't have any reverse video search services such as TinEye or Google's reverse image search.

notext.mp4

_{Video: https://user-images.githubusercontent.com/64683866/148960165-a210f2d2-6c41-4349-bd8d-a4cb673bc0af.mp4}

_{Source: https://www.youtube.com/watch?v=sJxDS1QInD0}

I have not created any reverse video search service but this package can be used to implement a scalable near-duplicate video retrieval (NDVR) system aka reverse video search service.

Videohash works by extracting frames at a default rate of 1 fps from videos then shrinking the frames to a 144x144 pixel square image and finally making a collage of these shrunk frames. The wavelet hash value of the collage is videohash value for the video.

Before going into further details about the working of videohash first let's notice the dissimilarities in the above two videos.

The original post by Ronaldo didn't have any text which was added by the person who made the meme. Also, notice the crops. The original video has an image of his child but the cropped video in the meme doesn't. Also, there are obvious changes in the aspect ratio because of the cropping.

Extraction of frames

Frames are extracted by the framesextractor.py module, which calls the FFmpeg to detect the amount of cropping to remove the black bar aka letterboxing. And again calls the FFmpeg to remove the black bars, extract frames and reduce the frame size to 144x144, these 3 tasks are done simultaneously.

Frames of the meme video.

Frames of the original video.

Making of collage

The collage of the frames is generated by the collagemaker.py module. It created the collage of the frames by sorting them according to the timestamp in the video and ensuring that the collage is square-shaped.

Collage generated from the frames of the meme video.

Collage generated from the frames of the original video.

Wavelet hash of collage

The final step is to calculate the hash value of the collage, which is in fact an image. We've reduced the problem of video hashing to that of the already solved problem of images hashing. The package calculates the wavelet hash value for the collage. Read more about wavelet hashing at https://fullstackml.com/wavelet-image-hash-in-python-3504fdd282b5.

Code

All the above stuff can be done using three lines of Python. Try:

import videohash

vh1 = videohash.VideoHash(url="https://user-images.githubusercontent.com/64683866/148959826-9b83c1ad-f9ef-4054-83df-d420ca318de6.mp4")

vh2 = videohash.VideoHash(url="https://user-images.githubusercontent.com/64683866/148960165-a210f2d2-6c41-4349-bd8d-a4cb673bc0af.mp4")

print(vh1 - vh2)

print(vh1 == vh2)

_{Run @ https://replit.com/@akamhy/ronaldo-videohash#main.py}

Output

0
True