Display Content Overlaying the Video: Profile name, Captions, Audio track name

[DouglasMartins1999]

This project is unique and amazing, thank you for your efforts! It would be great if, in addition to the video itself, part of the content that accompanies it on TikTok/Reels could also be displayed, such as:

• Description/caption
• Profile photo/name/link to that originated the post
• Name of the audio track used

Replicating part of the app's experience, and allowing you to use Erin as a backup. It may be possible to make Erin, even without a backend, search for a .txt/.json/.yaml file with the same name as the video and display the metadata in the interface.

[will-moss]

Hey,

Thanks a lot for the kind words, along with the detailed explanation of the feature.

I have been thinking about that feature for a long time, and gave up because I didn't know how to handle the "metadata" part. But you've just opened my eyes with the txt/json/yaml file. What a great idea.

I would like to ask your confirmation on the following flow before I implement it :

• The user opens Erin and logs in
• A video is played, the file's name is "video-345.mp4".
• In the background, the client (via JS) sends a request to Caddy, trying to fetch "video-345.json".
• Caddy replies with a file that has the following (made up) structure : https://pastebin.com/Sk367R2M
• The metadata (in JSON) is parsed by JS, and displayed on the user interface, trying to reproduce Tiktok's design.
• In the case that "video-xxx.json" doesn't exist, the client would simply keep playing the video, ignoring the "not found" error.

How does that look?

Thanks again!

[DouglasMartins1999]

Hi! Yeah, that's perfect for me like this. I would just like to ask a few questions to think about:

• Would there be some kind of loading placeholder while the metadata is loaded during video playback, or does the data just pop out into UI when loaded?

• To organize the files, would it be better for both the video and the .json to be in the same folder (for practical reasons), or to have separate folders since they are different information?

• Captions may be complex, including non-Latin characters, emojis, user mentions, hashtags (both are links, displayed in a different color), etc. Would this be difficult to replicate?

Thank you for your hard work, this will be a great feature for Erin!

[will-moss]

Re,

Thanks for your confirmation.

Regarding your questions :

• Data placeholder while loading : I think this is doable, but by default I had thought of the data just popping out into the UI. Since the UI is static (read, there's no layout reflow), the data would certainly just transition from invisible to visible, smoothly when it's loaded, and not trigger any reflow / movement of other parts.

• Files organization : I think any organization will work, whether it's one video per folder, or all videos in one. Erin will, anyway, recursively scan all the directories. The most important thing should be to give a unique name to every video, and have the matching metadata file somewhere (with the exact same name, just a different extension), with the exact same name, so they can be associated internally.

• Captions : The web client in Erin uses UTF-8 so, if I'm not mistaken, it should all work as expected with non-Latin characters, emojis, etc. However, user mentions and hashtags should certainly be encoded as raw HTML if you need the features you've mentioned (links, different colors, etc.). I believe the best way to satisfy all needs (present and future) is to allow raw HTML in the captions, so anyone can use their own formatting and style. And when no HTML is provided, the raw text will just be wrapped in a paragraph.

What do you think?

Thanks again for the kind words, along with the brainstorming!

[DouglasMartins1999]

Recursive search, independent folder organization, raw HTML for captions - it couldn't be any better. Thank you for clarifying, looking forward for the feature!

[will-moss]

Alright, it's released.

You should be able to pull the latest image, and have it working.

I've added a little paragraph in the README about it, but else, just create a matching .json file for your videos, using the following structure : https://github.com/will-moss/erin/blob/master/examples/video-metadata.json and it should work.

Let me know if you encounter any issue.

Note : I've tweaked the structure a little (from the one in the original Pastebin), because I realized we're not dealing with "audio" but with "videos", so.. just a naming thing!

[DouglasMartins1999]

Wow, that was pretty fast!
I just tested it, and it looks really cool!

Overall, I only have two concerns: Some captions can be very large and end up overlapping a large part of the video. I don't know if it would make sense to limit the height of the container and add a scroll bar to see the entire text.

I'm also a little concerned that the TikTok/Reels interface adds a transparent dark overlay to increase the contrast between the text and the video, and since we don't have it here, it might make the captions a little harder to read.

[will-moss]

Thanks a lot for your feedback, and thanks too for the screenshot with your concerns.

I have just published a new release that should address both issues you've mentioned.

Now, the bottom layer with metadata has a semi-opaque background (10% black, 90% transparent), and the caption has a maximum height of 50 pixels with a scrollbar when the content overflows.

Is it better now?

Thanks a lot once again, I'm so happy with that feature.

[will-moss]

Hi ! Just passing by to ask if I can close the issue, and most importantly, if everything works fine on your end.