Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
Huanshere committed Sep 15, 2024
1 parent dbeaf19 commit 8b48e79
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 29 deletions.
26 changes: 12 additions & 14 deletions README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@

</div>

## 🌟 What Can It Do
## 🌟 Our Vision

- 🍖 Fully automated video translation, generating Netflix-quality subtitles!
VideoLingo is an all-in-one video translation and localization tool designed to generate Netflix-quality subtitles. It aims to eliminate stiff machine translations and multi-line subtitles, enabling knowledge sharing across language barriers worldwide.

- 🎤 Clone your own voice for dubbing! (🚧 Still in development)
VideoLingo automates the entire process of video downloading, audio extraction, high-precision speech recognition, subtitle generation, high-quality text translation, subtitle alignment, and seamless integration of translated subtitles into the original video. With an intuitive Streamlit web interface, you can complete the entire workflow with just a few clicks, easily creating localized videos with Netflix-quality subtitles.

- ✨ Click-and-done in Streamlit!
VideoLingo is also actively developing voice cloning technology, which will soon support video dubbing, further enhancing the localization experience. Whether you're a content creator, educator, or someone with multilingual communication needs, VideoLingo can be a powerful assistant, helping you overcome language barriers and connect with global audiences.

> Check out the results! 💪
Expand All @@ -41,20 +41,22 @@ https://github.com/user-attachments/assets/25264b5b-6931-4d39-948c-5a1e4ce42fa7

## ✨ Features

- Subtitle segmentation using NLP and LLM
- Word-level timeline subtitle recognition using WhisperX

- Intelligent terminology knowledge base for context-aware translation
- Subtitle segmentation based on sentence meaning using NLP and GPT

- Three-step translation process: Direct Translation - Reflection - Improvement
- GPT-summarized intelligent terminology knowledge base for context-aware translation

- Precise word-level subtitle alignment
- Three-step translation process: Direct Translation, Reflection, and Paraphrasing to eliminate awkward machine translations

- Create 5 minutes of Netflix-quality bilingual subtitles for just 1 yuan
- Netflix-level single-line subtitle length and translation quality

- One-click integrated package launch, one-click video production in Streamlit!

- Developer-friendly: Step-by-step structured files for easy customization: [Chinese Technical Documentation](./docs/README_guide_zh.md) | [English Technical Documentation](./docs/README_guide_en.md)
> You can even run each `step__.py` file under `core` individually!
## 🏠 [Local Deployment Guide](./docs/install_locally_en.md)
## 🏠 [Local Installation Guide](./docs/install_locally_en.md)

## 🚧 Current Limitations

Expand Down Expand Up @@ -83,13 +85,9 @@ We are continuously improving VideoLingo, but there are still some limitations:

Thanks to the following open-source projects for their contributions:

- [whisper](https://github.com/openai/whisper): OpenAI's open-source automatic speech recognition system
- [whisper-timestamped](https://github.com/linto-ai/whisper-timestamped): Extension adding timestamp functionality to Whisper
- [whisperX](https://github.com/m-bain/whisperX): Extension adding timestamp functionality to Whisper
- [yt-dlp](https://github.com/yt-dlp/yt-dlp): Command-line tool for downloading YouTube videos and content from other websites
- [GPT-SoVITS](https://github.com/RVC-Project/GPT-SoVITS) & [GPT-SoVITS-Inference](https://github.com/X-T-E-R/GPT-SoVITS-Inference): Speech synthesis system and inference library based on GPT and SoVITS
- [FFmpeg](https://github.com/FFmpeg/FFmpeg): Complete, cross-platform solution for handling multimedia content
- [Ultimate Vocal Remover GUI v5 (UVR5)](https://github.com/Anjok07/ultimatevocalremovergui): Tool for separating vocals and instrumentals in music
- [json_repair](https://github.com/mangiucugna/json_repair): Super powerful library for repairing and parsing GPT's JSON output, seamlessly replacing json.loads

## 🤝 Contributions Welcome
Expand Down
28 changes: 13 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@

</div>

## 🌟 能做什么
## 🌟 初衷

- 🍖 全自动视频搬运工,生成 Netflix 品质的字幕!
VideoLingo 是一站式视频翻译本地化工具,旨在生成 Netflix 级别的高质量字幕,告别生硬机翻,告别多行字幕,让全世界的知识能够跨越语言的障碍共享。

- 🎤 克隆自己的声音进行配音!(🚧 仍在开发中)
VideoLingo 能够自动下载视频、提取音频、进行高精度的语音识别、生成字幕、执行高质量的文本翻译和字幕对齐,并将翻译后的字幕无缝集成到原始视频中。通过直观的 Streamlit 网页界面,只需点击几下就能完成整个流程,轻松创建出具有 Netflix 品质字幕的本地化视频。

- ✨ 在 streamlit 中点击-完成!
VideoLingo 还在积极开发声音克隆技术,很快将支持视频配音,进一步提升视频的本地化体验。无论是内容创作者、教育工作者还是多语言传播需求者,VideoLingo 都能成为强大的助手,帮助跨越语言障碍,连接全球观众。

> 看看效果吧!💪
Expand All @@ -41,21 +41,23 @@ https://github.com/user-attachments/assets/25264b5b-6931-4d39-948c-5a1e4ce42fa7

## ✨ 特点

- 使用 NLP 和 LLM 进行字幕分割
- 使用 WhisperX 进行单词级时间轴字幕识别

- 智能术语知识库,实现上下文感知翻译
- 使用 NLP 和 GPT 根据句意进行字幕分割

- 三步翻译过程:直接翻译 - 反思 - 改进
- GPT 总结智能术语知识库,实现上下文感知翻译

- 精确的单词级字幕对齐
- 三步直译、反思、意译,告别诡异机翻

- 仅需 1 元即可创作 5 分钟的 Netflix 级双语字幕
- Netflix 级别的单行字幕长度与翻译质量

- 一键整合包启动,在 streamlit 中一键出片!

- 开发者友好:逐步结构化文件,便于自定义开发 : [中文技术文档](./docs/README_guide_zh.md) | [英文技术文档](./docs/README_guide_en.md)
> 你甚至可以单独运行每一个`core`下的`step__.py`文件!

## 🏠 [本地部署教程](./docs/install_locally_zh.md)
## 🏠 [本地安装教程](./docs/install_locally_zh.md)

## 🚧 当前限制

Expand All @@ -82,15 +84,11 @@ https://github.com/user-attachments/assets/25264b5b-6931-4d39-948c-5a1e4ce42fa7

## 🙏 致谢

感谢以下开源项目的贡献:
感谢以下开源项目的贡献

- [whisper](https://github.com/openai/whisper): OpenAI的开源自动语音识别系统
- [whisper-timestamped](https://github.com/linto-ai/whisper-timestamped): 为Whisper添加时间戳功能的扩展
- [whisperX](https://github.com/m-bain/whisperX): 为Whisper添加时间戳功能的扩展
- [yt-dlp](https://github.com/yt-dlp/yt-dlp): 用于下载YouTube视频和其他网站内容的命令行工具
- [GPT-SoVITS](https://github.com/RVC-Project/GPT-SoVITS) & [GPT-SoVITS-Inference](https://github.com/X-T-E-R/GPT-SoVITS-Inference): 基于GPT和SoVITS的语音合成系统及推理库
- [FFmpeg](https://github.com/FFmpeg/FFmpeg): 用于处理多媒体内容的完整多平台解决方案
- [Ultimate Vocal Remover GUI v5 (UVR5)](https://github.com/Anjok07/ultimatevocalremovergui): 用于分离音乐中的人声和伴奏的工具
- [json_repair](https://github.com/mangiucugna/json_repair): 超无敌的 修复解析 gpt 的 json 输出的库,无缝替代 json.loads

## 🤝 欢迎贡献
Expand Down

0 comments on commit 8b48e79

Please sign in to comment.