AssemblyAI Captions Example

This open-source repo displays how to create video captions using VTT files exported from AssemblyAI's VTT endpoint!

In this app, we use a VTT file which has been generated by our VTT endpoint to display video captions in the browser. This app uses the subtitle package to parse the VTT file so the text could be added to the video.

How to Install and Run the Project Locally

1. Clone the repo to your local machine.
2. Open the terminal in the root directory of the project.
3. Run npm install to install all dependencies.
4. Run npm run start to start the server (will run on http://localhost:8000).
5. Open another terminal and run npm run client to start the client side (will run on http://localhost:3000)
6. Navigate to http://localhost:3000 in your browser and play the video to see live captions!

VTTCues

In order to display captions on the video, we need to parse the VTT file and retrieve cues to add to the video track. We use the subtitle package for this. The exact method used to parse the VTT file is as follows:

The above code reads the VTT file, parses the data and returns an array of cues. A cue is data that contains the start time, end time, and text of each caption. We add each of these cues to the TextTrack of the video by using the VTTCue interface. It should be noted that the subtitle package returns the timestamps in milliseconds and VTTCues reads timestamps in seconds and fractions of a second. In this repo, the subtitle timestamps have been converted to seconds in order to get the captions to display correctly.

Further Documentation

• AssemblyAI API

• Subtitle

• TextTrack

• VTTCue interface

Contact Us

If you have any questions, please feel free to reach out to our Support team - support@assemblyai.com!