Skip to content

Conversation

SHIVAJIKUMAR007
Copy link
Contributor

@SHIVAJIKUMAR007 SHIVAJIKUMAR007 commented Aug 24, 2022

this is the video effect registry
to create a video effect you have to implement VideoFrameProcessorFactoryInterface

for example :

import org.webrtc.*;
import com.oney.WebRTCModule.videoEffect.VideoFrameProcessor;
import com.oney.WebRTCModule.videoEffect.VideoFrameProcessorFactoryInterface;

public class Rotate90Factory implements VideoFrameProcessorFactoryInterface {

    public VideoFrameProcessor build() {
        return new VideoFrameProcessor() {
            @Override
            public VideoFrame process(VideoFrame frame, SurfaceTextureHelper textureHelper) {
                return new VideoFrame(frame.getBuffer().toI420(), 0, frame.getTimestampNs());
            }
        };
    }
}

then you have to add this processor in processor map

import com.oney.WebRTCModule.videoEffect.ProcessorMap;

@ReactMethod
public void addMethods() {
    Rotate90Factory rotate90 = new Rotate90Factory();
    ProcessorMap.addProcessor("rotate90", rotate90);
    ProcessorMap.addProcessor("rotate270", () - > (VideoFrameProcessor)(frame, textureHelper) - > new VideoFrame(frame.getBuffer().toI420(), 180, frame.getTimestampNs()));
}

your videoEffect is registered now

to apply this videoEffect you have to call mediaStreamTrack._setVideoEffect(name);
and to remove this effect you just call mediaStreamTrack._setVideoEffect(null);

const onRegisterVideoEffect = () => {
    stream.getVideoTracks().forEach(track => {
        track._setVideoEffect('rotate90');
    });
};

const onDiscardVideoEffect = () => {
    stream.getVideoTracks().forEach(track => {
        track._setVideoEffect(null);
    });
};

Copy link
Member

@saghul saghul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking much better! Please take a look and my comments and make sure to add documentation to every new class and method describing what it does and what the parameters are and do.

@saghul
Copy link
Member

saghul commented Aug 29, 2022

Can you please paste a link to the blur example?

Copy link
Member

@saghul saghul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM can you have one final look @tmoldovan8x8 ?

@saghul
Copy link
Member

saghul commented Sep 12, 2022

@SHIVAJIKUMAR007 don't we need to release the frame if outputFrame is null ?

@SHIVAJIKUMAR007
Copy link
Contributor Author

SHIVAJIKUMAR007 commented Sep 12, 2022

yes, you are right.

This work was done during Google Summer of Code 2022 under the Jitsi
organization. Mentors: @saghul and @tmoldovan8x8.
@saghul saghul force-pushed the videoEffectRegistry branch from 0198ba4 to 74b1e77 Compare October 4, 2022 08:57
@saghul saghul merged commit 94b1060 into react-native-webrtc:master Oct 4, 2022
@saghul
Copy link
Member

saghul commented Oct 4, 2022

Congrats @SHIVAJIKUMAR007 this is now merged and will be part of the next release!

@SHIVAJIKUMAR007
Copy link
Contributor Author

SHIVAJIKUMAR007 commented Oct 4, 2022

Thanks, @saghul and @tmoldovan8x8 for all the guidance and support.
For queries please mail me: official.shivaji007@gmail.com

@fukemy
Copy link

fukemy commented Jan 31, 2023

really good, thannk so much, @SHIVAJIKUMAR007 , could you please share some processor to blur the person background for example?

@SHIVAJIKUMAR007
Copy link
Contributor Author

really good, thannk so much, @SHIVAJIKUMAR007 , could you please share some processor to blur the person background for example?

you can check my repo : https://github.com/SHIVAJIKUMAR007/real-time-VideoProcessing/tree/master/android/app/src/main/java/com/vchat/backgroundEffect

@darklordemperor
Copy link

@SHIVAJIKUMAR007 , can i ask about where localtion i need to implement VideoFrameProcessorFactoryInterface , or can just implement to android native code in project along @reeactMethod? because my plan are add background with custom image by filter with Ai segmention(i use tensorflow lite AI) , can your tell me logic to handle my case? thank!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants