(click image to enlarge)
This project provides a collection of Real-ESRGAN Compact ONNX upscaling models, along with a custom build of mpv video player. The video player (currently Windows only), enables real-time upscaling of 1080p content to 4K by running these models using TensorRT (NVIDIA only) or DirectML (for AMD or Intel Arc). While the default configuration upscales using the 2x_AnimeJaNai models, it can be easily customized to utilize any Real-ESRGAN Compact ONNX models.
Join the JaNai Discord server to get the latest news, download pre-release and experimental models, get support and ask questions, share your screenshots (use the s key in mpv), or share your feedback. 日本語も大丈夫です。
Ensure your NVIDIA graphics drivers are up to date. Download and extract the latest release archive of mpv-upscale-2x_animejanai. Open the video player at mpvnet.exe.
When playing a video for the first time, a TensorRT engine file will be created for the selected ONNX model. Playback will be paused and a command prompt box will open. Please make sure to wait while the engine is created. Engine creation only needs to happen once per model. Playback will resume on its own when finished.
To confirm upscaling status, press ctrl+J to view upscaling stats. This shows the current profile, and the currently running upscaling models if any.
The player is preconfigured to upscale with 2x_AnimeJaNai models, and makes 3 upscaling profiles available by default. The available profiles are described in more detail below. Any of these profiles can be selected on the fly using the keybinding listed below.
| Profile | Description | Keybinding | Minimum recommended GPU for upscaling 1080p to 4k |
|---|---|---|---|
| Quality | Highest quality model | Shift+1 |
RTX 4090 |
| Balanced | High quality model which trades slight quality for major performance gains | Shift+2 |
RTX 3080 |
| Performance | Fastest performance model which sacrifices a bit more quality | Shift+3 |
RTX 3060 |
The default upscaling profile is the Balanced profile which is recommended for users running an NVIDIA RTX 3080 or higher.
Upscaling can be further customized using the AnimeJaNaiConfEditor which can be launched by pressing ctrl+E from mpvnet. The editor allows the setup of up to 9 custom slots and also the use of custom chains, conditional settings based on video resolution and framerate, downscaling to improve performance, and more. The default upscaling profile can also be set using the conf editor.
All other mpv settings can be configured by editing mpv-upscale-2x_animejanai/portable_config/mpv.conf (see the mpv manual for all options) for mpv options or mpv-upscale-2x_animejanai/portable_config/input.conf for mpv keybindings.
By default, screenshots can be taken with the s key and are stored in mpv-upscale-2x_animejanai/portable_config/screenshots.
mpv-upscale-2x_animejanai is configured to use TensorRT by default for optimal performance, but TensorRT requires an NVIDIA GPU. Users with AMD or Intel Arc GPUs can use DirectML instead. See the wiki page for detailed instructions.
The 2x_AnimeJaNai models are a collection of real-time 2x Real-ESRGAN Compact, UltraCompact, and SuperUltraCompact models designed specifically for doubling the resolution of HD and SD models.
Most HD anime are not produced in native 1080p resolution but rather have a production resolution between 720p and 1080p. When the anime is distributed to consumers via TV broadcast, web streaming, or home video, the video is scaled up to 1080p, leading to scaling artifacts and a loss of image clarity in the source video. The aim of these models is to address these scaling and blur-related issues while upscaling to deliver a result that appears as if the anime was originally mastered in 4K resolution.
The development of the V3 models spanned over seven months, during which over 100 release candidate models were trained and meticulously refined. The V3 models introduce several notable improvements compared to their V2 counterparts, including:
- More faithful appearance to original source
- Improved handling of oversharpening artifacts, ringing, aliasing
- Better at preserving intentional blur in scenes using depth of field
- More accurate line colors, darkness, and thickness
- Better preservation of soft shadow edges
Overall, the V3 models yield significantly more natural and faithful results compared to the V2 models.
2x_AnimeJaNai SD V1 models are in developmnent. The latest release of mpv-upscale-2x_animejanai includes an early beta model for 2x_AnimeJaNai SD V1. While the 2xAnimeJaNai HD models can also work well for some SD sources, those models were specifically trained to upscale HD anime and don't always work well for SD sources. The SD models are designed to upscale SD anime to appear as if the anime was mastered in HD resolution. With sufficient hardware, these models can be stacked with the HD models to upscale SD anime to 4k resolution.
Benchmarks for various hardware configurations tested against various upscaling configurations are available on the wiki.
Any media player which supports external DirectShow filters should be able to run these models, by using avisynth_filter to get VapourSynth running in the video player.
The 2x_AnimeJaNai_V2 ONNX models can be used on a PC with any graphics card to render upscaled videos, even when using graphics cards not fast enough for realtime playback. Please see the AnimeJaNaiConverterGui project to create upscaled video files using a Windows GUI. Other options include chaiNNer or VSGAN-tensorrt-docker, which are multiplatform options for Windows and non-Windows users.
- MangaJaNai: Upscale manga with ESRGAN models
- VideoJaNai: Windows GUI for upscaling videos with extremely fast performance
- traiNNer-redux: Software for training upscaling models
- Upscale Wiki and associated Discord server
- 422415 for significant assistance in dataset preparation and continuous feedback during development of V2 models
- Community feedback on V1 models
- MPV_lazy and vs-mlrt
- traiNNer-redux
- Dataset Destroyer
- Real-ESRGAN
- OpenModelDB
- getnative and anibin
