Recently, I downloaded an old documentary made in early 2000s. The original resolution was 854×480 with a less than 1000 KB/S bitrate. Because I like the content so there is enough reason to enhance my watching experience.

Download installer instead of the zip pack from k4yt3x’s github repo, this will install all binaries and dependencies for Windows.

After restart the system, open video2x and load a video, it can automatically detect the GPU if it exists. Gaming GPU with higher frequency/hashrate is preferred over larger VRAM in this case.

There are two processing modes, resolution upscaling and frame interpolation. We can upscale first and then increase the frame rate.

Under upscaling mode, we have three filters avalible. These are similar to the image tools such as stable diffusion upscale or the image upscaler tool upscayl because a video is basically a sequence of images.

Model selection

  • Real-ESRGAN: primarily aimed at illustrations and anime, but has a variation for real photographs.
  • realesr-animevideov3: the fastest and most accurate for anime/vector
  • realesrgan-plus-anime: alternative apporach for anime/vector
  • realesrgan-plus: general-purpose model for both photos and illustrations
  • Real-CUGAN: specialized for illustrations and anime with gradients, no good for photos
  • models-pro: the most accurate with versatile options
  • models-nose: more denoise and sharpen edges, loses detail.
  • models-se: alternative apporach to models-pro
  • libplacebo: uses Anime4K GLSL shaders exclusively for anime

As I’m working with historical footage, realesrgan-plus is the only option for the task (locked 4x upscale, but can be compressed to 1080p later with ffmpeg). For Encoder Options, I’m using .mp4 suffix.

Under frame interpolation mode, I’m using the default RIFE v4.26 option without any problem.

As a result, the average bitrate bumped up from 800 KB/S to 30 MB/S and the files size from 480p(500MB) to 4k(12GB) and 4k60f(16GB).

The upscale process takes about 7days and 1 day for interpolation.

The 4K60fps video plays good on machines with good GPU, but on some lower-end machines I got a lot of stuttering and lag.

Many factors can overcome this problem such as switching player, codecs and decoders. This reminds me back in the 2000s, when Media Player Classic forks and KMPlayer mods are the coolest software among P2P/HTPC community.

But things are much simpler nowadays with MPV, by switching off hardware decoding (Ctrl+H) the problem disappears. To make this option as default, passing --hwdec=no in the cli or add hwdec=no line into /home/username/.config/mpv/mpv.conf.