Frame Data

[MohamedHassanKhames]

Well, I'm using your library to display a video stream and it's working properly. However, I want to get the actual data from the frames but it returns null.
Is there a way to get the data?
Thanks.

`
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(SOURCE_UDP);
grabber.start();
Frame frame = null;
while ((frame = grabber.grab()) != null ) {
ByteBuffer data = frame.data; // return null

}

`

[saudet]

The image data is in Frame.image[0].

[MohamedHassanKhames]

Yeah, I managed to get this but I thought I could combine the whole video data both image and sound.
Is this possible?

[MohamedHassanKhames]

Basically, I want to get the video data for a specific number of frames or seconds to be able to store it so if you can help I would really appreciate it.

[saudet]

Sure, the data for audio frames are in Frame.samples.

[MohamedHassanKhames]

How can I use it to capture the frame from the grabber?

[saudet]

You don't, you use a FrameGrabber such as FFmpegFrameGrabber for that, as per the Demo class in the README.md file:
https://github.com/bytedeco/javacv/#sample-usage

[MohamedHassanKhames]

Well, as you can see here in my main function I'm trying to grab the video and run it using FFmpegGrabber and then try to record it and save it using the FFmpegFrameRecorder. However, the output video is not working properly as per the images but it's working fine regarding the audio so do you have any idea why?

` public static void main(String[] args) throws UnknownHostException, FileNotFoundException, Exception, FrameRecorder.Exception {

    FFmpegFrameGrabber grabber = new FFmpegFrameGrabber("C:\\Users\\mohamad\\My_Stuff\\Utilities\\Adele.mp4");
    File file = new File("C:\\Users\\mohamad\\My_Stuff\\Mongo\\framerecoder.mp4");
    //OutputStream out = new FileOutputStream("C:\\Users\\mohamad\\My_Stuff\\Mongo\\framerecoder.mp4");
    FFmpegFrameRecorder frameRecorder = new FFmpegFrameRecorder(file,1920,1080);

    try {

        grabber.start(true);
        CanvasFrame canvas = new CanvasFrame("VideoCanvas",CanvasFrame.getDefaultGamma() / grabber.getGamma());
        
        final AudioFormat audioFormat = new AudioFormat(grabber.getSampleRate(), 16,grabber.getAudioChannels(),true, true);
        final DataLine.Info info = new DataLine.Info(SourceDataLine.class,audioFormat);
        final SourceDataLine soundLine = (SourceDataLine) AudioSystem.getLine(info);
        soundLine.open(audioFormat);
        soundLine.start();

        ExecutorService executor = Executors.newSingleThreadExecutor();
        
        frameRecorder.setAudioChannels(grabber.getAudioChannels());
        frameRecorder.setVideoCodec(grabber.getVideoCodec());
        frameRecorder.setFrameRate(grabber.getFrameRate());
        frameRecorder.setVideoOption("preset", "ultrafast");
        frameRecorder.setVideoQuality(0.1);

        int k = 0 ;
        Frame frame = null;
        frameRecorder.start();

        while ((frame = grabber.grab()) != null && k < 251) {
            
            if (frame.image != null){
                k++;
            }
            
            // grabbing the image and show it
            canvas.setCanvasSize(grabber.getImageWidth(), grabber.getImageHeight());
            canvas.showImage(frame);

            // grabbing the sound and run it
            if (frame.samples != null) {

                final ShortBuffer channelSamplesShortBuffer = (ShortBuffer) frame.samples[0];
                channelSamplesShortBuffer.rewind();
                final ByteBuffer outBuffer =
                        ByteBuffer.allocate(channelSamplesShortBuffer.capacity() * 2);
                for (int i = 0; i < channelSamplesShortBuffer.capacity(); i++) {
                    short val = channelSamplesShortBuffer.get(i);
                    outBuffer.putShort(val);
                }
                /**
                 * We need this because soundLine.write ignores
                 * interruptions during writing.
                 */
                try {
                    executor.submit(new Runnable() { public void run() {
                        soundLine.write(outBuffer.array(), 0, outBuffer.capacity());
                        outBuffer.clear();
                    }}).get();
                } catch (InterruptedException | ExecutionException interruptedException) {
                    Thread.currentThread().interrupt();
                }
            }
            
             /// recoder the frame after playing it. 
            frameRecorder.setTimestamp(frame.timestamp);
            frameRecorder.record(frame);

        }
        
        frameRecorder.setCloseOutputStream(true);
        frameRecorder.stop();
        grabber.stop();


    } catch (Exception | LineUnavailableException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }finally {
    if (grabber != null) {
        grabber.close();
        frameRecorder.close();
        System.exit(0);
    }
    }

}`

[MohamedHassanKhames]

I only need to record the video as a file or output stream I don't even need to play it so if it is possible I'd be grateful for your assistance.

[MohamedHassanKhames]

Well, it worked properly with a saved video on my pc but when I tried it with a UDP stream it didn't work although I can see the image while playing.
Any ideas why?

[saudet]

UDP isn't a reliable protocol, use TCP.

[MohamedHassanKhames]

I tried it with http and still the same issue :(

[saudet]

Don't use OutputStream, it's not something that works well with the MP4 format.

[MohamedHassanKhames]

What should I use then?

[saudet]

Just the URL itself.