Manual frameset creation

[jc211]

Hi,
I have one depth frame and one color frame. I want to create a frameset myself of these two. Is this possible?

[MartyG-RealSense]

There was a previous case in 2019 where someone tried to create framesets manually using software device. Their work may provide a useful reference for you.

https://support.intelrealsense.com/hc/en-us/community/posts/360037090133-Construct-Frameset-Object

[dorodnic]

Hi @jc211 @MartyG-RealSense
Another way to do this is using custom processing block:

std::vector<rs2::frame> bundle;
rs2::processing_block bundler([&](rs2::frame f, rs2::frame_source& src) {
    bundle.push_back(f);
    if (bundle.size() == 2)
    {
        auto fs = src.allocate_composite_frame(bundle);
        src.frame_ready(fs);
        bundle.clear();
     }
});
rs2::frame_queue q;
bundler.start(q);

Use like this:

bundler.invoke(f.get_depth_frame());
bundler.invoke(f.get_color_frame());
rs2::frameset fs = q.wait_for_frame();

In either of the methods you need some object to own any frames you want to create (either software device or processing block). This is because all frames in the SDK are recycled for performance reasons, so they must be part of some larger class.

[jc211]

@dorodnic Thanks, that works!

[wangbz98]

Hi @jc211 @MartyG-RealSense Another way to do this is using custom processing block:

std::vector<rs2::frame> bundle;
rs2::processing_block bundler([&](rs2::frame f, rs2::frame_source& src) {
    bundle.push_back(f);
    if (bundle.size() == 2)
    {
        auto fs = src.allocate_composite_frame(bundle);
        src.frame_ready(fs);
        bundle.clear();
     }
});
rs2::frame_queue q;
bundler.start(q);

Use like this:

bundler.invoke(f.get_depth_frame());
bundler.invoke(f.get_color_frame());
rs2::frameset fs = q.wait_for_frame();

In either of the methods you need some object to own any frames you want to create (either software device or processing block). This is because all frames in the SDK are recycled for performance reasons, so they must be part of some larger class.

Hi! Can this part of codes be transformed into python language?

[MartyG-RealSense]

Hi @wangbz98 The custom processing block is supported in Python. The official pyrealsense2 documentation for this function is in the link below.

https://intelrealsense.github.io/librealsense/python_docs/_generated/pyrealsense2.processing_block.html#pyrealsense2.processing_block

[wangbz98]

THX! But I'm not able to rewrite it with python, could you please help me?

[MartyG-RealSense]

I researched your question carefully but there are no scripting references available for processing_block on Python other than the documentation page that I linked to, unfortunately.

[DonLuyendijk]

Hi @MartyG-RealSense ,

I would also like to use the custom processing block example in Python. I tried to translate it, but I can't get it to work.
I made a script that should create a frameset from a color frame and depth frame. The script crashes at the q.wait_for_frame() call, with the message RuntimeError: Frame did not arrive in time!. I then added a print call to the bundler callback, but nothing is printed. It seems like the callback is never called when bundler.invoke() is called.
Can i get some help with this issue?

Here's the script:

import pyrealsense2 as rs

def bundler_callback(f, src):
	print('bundler function called')
	bundle.append(f)
	if len(bundle) == 2:
		fs = src.allocate_composite_frame(bundle)
		src.frame_ready(fs)
		bundle.clear()

pipe = rs.pipeline()
cfg = rs.config()
cfg.enable_device_from_file('stairs.bag')
pipe.start(cfg)

fs = pipe.wait_for_frames()

bundle = list()
bundler = rs.processing_block(bundler_callback)
q = rs.frame_queue()
bundler.start(q)

bundler.invoke(fs.get_color_frame())
bundler.invoke(fs.get_depth_frame())

fs = q.wait_for_frame()

And here's a link to the sample file I used:
https://librealsense.intel.com/rs-tests/TestData/stairs.bag

[MartyG-RealSense]

Hi @DonLuyendijk There are not any references about custom processing blocks in Python that I can point you to. Looking at your script though, I would suspect that you may need to place the name of your defined callback within the pipe start instruction's brackets. A Python example of this is demonstrated in a script at #5417

For example: pipe.start(cfg, bundler_callback)