24 to 60 fps?

[spyro2000]

Hi there,

RIFE looks fantastic, but as fair as I know I only can enter integer numbers as scale factor, correct? So when I want to interpolate 24 fps to 60 (by far the most common case I suppose) I know no other way than interpolating to 120 (factor 5) and then drop any other frame to get 60.

But even that doesn't seem to be possible as supported scale factors are only 2x, 4x, 8x (no 5x option).

So, is RIFE able to make 24 fps movie content run smooth on 60 Hz displays?

[hzwer]

So, is RIFE able to make 24 fps movie content run smooth on 60 Hz displays?
It has not been implemented at present. We can only approximate frame extraction after multiple interpolation of frames.

[vadash]

24 -> rife x2 -> 48 -> +4% speed up with mpv -> 50 fps @ 50 hz display

Both my TV and monitors works fine with 50 hz but refuses to switch to 48 hz

[spyro2000]

There is simply not standard to display 48 fps on a display (only in Peter Jackson's head). That's why 60 fps for everything would be the best solution, but tell that to directors ;)

A speedup was common in past days to convert FILM content to PAL but that should be a thing of the past (you also have to speedup audio).

Guess we just have to wait until RIFE 2 to make this really useful. Looking forward! :)

[vadash]

+4% is nothing and dont affect user. Yep audio gets +4% too ofc

# "ReClock" replacement
video-sync=display-resample
video-sync-max-video-change=25

We are really close to realtime btw

its fp32 and fp16 models can get x2 boost

[Mr187k]

@vadash I'm running MadVR with Potplayer. Is there any way to make it play +4%? Running 48fps on 50Hz display has slight hiccups every second.
Btw with which GPU are you getting almost realtime performance?

[vadash]

@vadash I'm running MadVR with Potplayer. Is there any way to make it play +4%? Running 48fps on 50Hz display has slight hiccups every second.
Btw with which GPU are you getting almost realtime performance?

reclock for x86

[adahbingee]

This seems to be the training probem and model design for RIFE.
The simple 2x interpolate strategy makes it easy to find any video for training with dropping out every odd/even frames.
I'm also wondering what if RIFE can interpolate any phases in two frame not just phase 0.5.
If I do somthing time interpolate trick on flow output works?
@hzwer

[hzwer]

@adahbingee Unfortunately, we have not found a simple strategy to expand RIFE to such a scenario. DAIN, BMBC and EDSC are recommended.

[spyro2000]

Hmm, ok. Thanks for being this honest. Guess I'll have to wait a few more years. ;)

[hzwer]

We support it in our new version paper, the model may be released within 1 month.

[spyro2000]

Wohooo, that came out of nowhere! Can't wait! :)))

[spyro2000]

Hi @hzwer, really looking forward, any news on this? :)

[vadash]

Is it worth it ?
60hz: 12 original + 48 interpolated
50hz: 24 original + 24 interpolated (and +5% speedup)

So 60 hz is twice as slow and have less OG frames. Just use 50 hz

[hzwer]

@spyro2000 Please check https://github.com/hzwer/Practical-RIFE/blob/6ae037d98b671f1221551e2bc819e8e2370f0aa2/inference_video.py#L174, we have already supported this feature.
I don't know if the downstream software is developed.
@vadash Some display devices cannot directly support the 50hz format, maybe this is the important reason for this feature.

[spyro2000]

Pretty much any display runs on 60 Hz, especially PC, Notebooks and Smartphones by default. And even if it would support 50 Hz, browser, apps and videoplayer don't change the dispay's framerate.

BTW: I don't think that 60 fps out of 24 is "12 real + 48 interpolated" but all (!) of them because of different time stepping. That's what makes it more difficult.

Better TVs can do this for years by calculating and interpolation motion vectors but I would love to see that in RIFEs high quality. :)

I am a little bit confused right know about the current state. So can RIFE now convert from 24 to 60 Hz in it's current version? Is there any documentation how this can be done? Thank you. :)

[mirh]

Pretty much any display runs on 60 Hz, especially PC, Notebooks and Smartphones by default.

Pretty much any display can be forced to run at 50Hz then (droids may have been a bit trickier tbh, but today it's pretty straightforward too)

And even if it would support 50 Hz, browser, apps and videoplayer don't change the dispay's framerate.

Apps and videoplayers totally can change the refresh rate. Of course they must be programmed to do so (like most games trivially are).
Then browsers aren't ever going to support full mode changes for a number of reasons, but soon enough you should be able to do everything with VRR (which can also be hacked into some older monitor btw).

So can RIFE now convert from 24 to 60 Hz in it's current version?

It is my understanding that the RIFEm model (that they added with their november paper) should support arbitrary interpolation factors.
I'm unsure if they still nevertheless need to be integers, or 2.5 could be also possible.

[spyro2000]

Pretty much any display can be forced to run at 50Hz

Yeah, good luck doing that from within a browser. ;)

Apps and videoplayers totally can change the refresh rate. Of course they must be programmed to do so (like most games trivially are).

But they don't. And even if they are, they are not configured like that.

It is my understanding that the RIFEm model (that they added with their november paper) should support arbitrary interpolation factors.

Highly appreciated :)

[mirh]

But they don't.

They do, and people already told you about reclock here.
MadVR and mpv are far better and more modern examples.

Yeah, good luck doing that from within a browser. ;)

w3c/html#375 (comment)
Even though (after checking more thoroughly) I'll grant to you that the ways of improving the standards of the web are immensely long and wide.

[spyro2000]

Seriously, I do not want to speedup a movie by 4% and then use Reclock or a VRR-Mode for browser or something just to view a movie without stutter. I did that 15 years ago and it did not work out at the long run, believe me. ;)

Instead, thats an excellent use case for frame interpolation. I just could convert a 24 fps source before uploading it to YouTube or sharing it otherwise. Everyone gets fluid motion on their display with default settings. So I highly appreciate it, that RIFE will be able to convert 24 fps to 60 fps in the future, looking forward. ;)

[vadash]

RIFE will be able to convert 24 fps to 60 fps

A lot of tools already can do that

Flowframes too

[mirh]

Nobody talked about speedups (which of course suck)... and insofar as what I said about RIFE may support what you are looking for, that should be already a thing (e.g. in flowframes it's known as rife-v4)

[spyro2000]

Thank you, I know about Flowframes and I regularly use SVP, but RIFE takes that on another level, quality-wise.