The right way to deproject pixel to point with align to depth frame. #5403

BadMachine · 2019-12-08T13:58:41Z

Required Info
Camera Model	D435i
Firmware Version	(2.29.0 ?)
Operating System & Version	Windows 10
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC
SDK Version	latest
Language	{C++}
Segment	{VR/AR }

Issue Description

I'm trying to get accurate points in 3d space using "rs2_deproject_pixel_to_point" function.
I have 2 problems with it:

Sometimes, when i have to get distance to {x, y} coordinate of pixel i`m getting distance = 0, as you can realize then result of "rs2_deproject_pixel_to_point" is {x=0, y=0, z=0}
When im getting non-zero values of depth (in meters) the values can be in range (true, close to be true and absolutly inaccurate), i mean the distance value is jumping over 1-2 meters.

Steps of the programm

Grabbing frames and intrinsics

       rs2::pipeline p;
       rs2::config cfg;
		cfg.enable_stream(RS2_STREAM_DEPTH, 848, 480, RS2_FORMAT_Z16, 30);
		cfg.enable_stream(RS2_STREAM_COLOR, 848, 480, RS2_FORMAT_RGB8);
       p.start(cfg);
while (!QThread::currentThread()->isInterruptionRequested())
		{	
			rs2::frameset frames = p.wait_for_frames();
			frames = align_to_depth.process(frames);
			rs2::depth_frame depth = frames.get_depth_frame();
			auto inrist = rs2::video_stream_profile(depth.get_profile()).get_intrinsics();
			...

		}

Getting necessary pixels in frame

Finding 3d coordinates of those pixels

			float planarPoint3d[3];
			int x = toSend.captured[i.key()].x;
                        int y = toSend.captured[i.key()].y;

float pixel_distance_in_meters = depth.get_distance(x, y);
rs2_deproject_pixel_to_point(planarPoint3d, &inrist, {float(x), float(y)}, pixel_distance_in_meters);

RESULT

Final questions

Am i using those function right?
Did i make mistake when align color frame to depth?
Did i get wrong intrinsics?
What am i doing wrong?)

BadMachine · 2019-12-08T19:21:26Z

UPD:
i applied few filters, like decimation, spatial and temporal, and got better, but still sometimes it`s "jumping" and shows wrong depth values

ev-mp · 2019-12-09T10:34:06Z

@BadMachine , getting the final result oscillating in range of +-1m is unacceptable, so it is more likely that the data path is not configured correctly.
There is always a possibility that the device is not functioning properly, so running the rs-depth-quality tool and using the accompanying white paper allows to assess whether the depth characteristics are in line with the expected results

As for the reported behavior -
Having zero depth value is a legitimate input, it just mean that that the sensor was not able to find a high-confidence match for the specific [i,j] coordinate.The mentioned tool rs-depth-quality has a similar metric called "Fill Rate" that shows the percentage of valid (non-zero) pixels in frame, and the number is generally less than 100%.
There are methods to improve the depth quality that affect the fill rate , such as adjusting preset, turning on and modifying the laser power. Reading the relevant white paper can be a good reference point.
One of the ways to reduce the number of zero points is to apply Depth decimation filter which in addition to reducing the depth resolution also delivers some "hole filling" capability.

But in any event these steps are secondary to the actual verification of the user c++ code.

I'll make certain assumptions as the snippet is neither covering the configuration part nor the RGB features/image handling. So the things to verify are:

The alignment is from color to depth as the member align_to_depth suggests, i.e. you generate a synthetic RGB frame captured as if it were positioned in the Depth sensor's location. If that is correct then the intrinsic values you extract are correct as well.
The RGB features that you extract for the skeleton tracking area calculated based on the "Aligned RGB" and not the original RGB frame.

ev-mp · 2019-12-09T10:47:20Z

@BadMachine , looking deeper at the posted results of deprojected data I can see the depth fluctuates in range of [1.368-1.413] meter ,i.e. it seem to be +-2.5cm which approximates for 1.8% of the range.
Note that that the D435i specification states RMS<2%, so the presented values are within the boundaries.

Can you provide an example with valid depth values deviating within 1 meter range?

BadMachine · 2019-12-09T11:15:49Z

@ev-mp Thank you for your reply!
First of all:
I applied "custom preset":

{
    "aux-param-autoexposure-setpoint": "1536",
    "aux-param-colorcorrection1": "0.298828",
    "aux-param-colorcorrection10": "-0",
    "aux-param-colorcorrection11": "-0",
    "aux-param-colorcorrection12": "-0",
    "aux-param-colorcorrection2": "0.293945",
    "aux-param-colorcorrection3": "0.293945",
    "aux-param-colorcorrection4": "0.114258",
    "aux-param-colorcorrection5": "-0",
    "aux-param-colorcorrection6": "-0",
    "aux-param-colorcorrection7": "-0",
    "aux-param-colorcorrection8": "-0",
    "aux-param-colorcorrection9": "-0",
    "aux-param-depthclampmax": "65536",
    "aux-param-depthclampmin": "0",
    "aux-param-disparityshift": "0",
    "controls-autoexposure-auto": "True",
    "controls-autoexposure-manual": "5040",
    "controls-color-autoexposure-auto": "True",
    "controls-color-autoexposure-manual": "156",
    "controls-color-backlight-compensation": "0",
    "controls-color-brightness": "0",
    "controls-color-contrast": "50",
    "controls-color-gain": "64",
    "controls-color-gamma": "300",
    "controls-color-hue": "0",
    "controls-color-power-line-frequency": "3",
    "controls-color-saturation": "64",
    "controls-color-sharpness": "50",
    "controls-color-white-balance-auto": "True",
    "controls-color-white-balance-manual": "4600",
    "controls-depth-gain": "16",
    "controls-laserpower": "300",
    "controls-laserstate": "on",
    "ignoreSAD": "0",
    "param-amplitude-factor": "0",
    "param-autoexposure-setpoint": "1536",
    "param-censusenablereg-udiameter": "8",
    "param-censusenablereg-vdiameter": "9",
    "param-censususize": "8",
    "param-censusvsize": "9",
    "param-depthclampmax": "65536",
    "param-depthclampmin": "0",
    "param-depthunits": "1000",
    "param-disableraucolor": "0",
    "param-disablesadcolor": "0",
    "param-disablesadnormalize": "0",
    "param-disablesloleftcolor": "0",
    "param-disableslorightcolor": "1",
    "param-disparitymode": "0",
    "param-disparityshift": "0",
    "param-lambdaad": "751",
    "param-lambdacensus": "6",
    "param-leftrightthreshold": "10",
    "param-maxscorethreshb": "2893",
    "param-medianthreshold": "796",
    "param-minscorethresha": "4",
    "param-neighborthresh": "108",
    "param-raumine": "6",
    "param-rauminn": "3",
    "param-rauminnssum": "7",
    "param-raumins": "2",
    "param-rauminw": "2",
    "param-rauminwesum": "12",
    "param-regioncolorthresholdb": "0.784736",
    "param-regioncolorthresholdg": "0.565558",
    "param-regioncolorthresholdr": "0.985323",
    "param-regionshrinku": "3",
    "param-regionshrinkv": "0",
    "param-robbinsmonrodecrement": "25",
    "param-robbinsmonroincrement": "2",
    "param-rsmdiffthreshold": "1.65625",
    "param-rsmrauslodiffthreshold": "0.71875",
    "param-rsmremovethreshold": "0.809524",
    "param-scanlineedgetaub": "13",
    "param-scanlineedgetaug": "15",
    "param-scanlineedgetaur": "30",
    "param-scanlinep1": "155",
    "param-scanlinep1onediscon": "160",
    "param-scanlinep1twodiscon": "59",
    "param-scanlinep2": "190",
    "param-scanlinep2onediscon": "507",
    "param-scanlinep2twodiscon": "493",
    "param-secondpeakdelta": "647",
    "param-texturecountthresh": "0",
    "param-texturedifferencethresh": "1722",
    "param-usersm": "1",
    "param-zunits": "1000",
    "stream-depth-format": "Z16",
    "stream-fps": "30",
    "stream-height": "720",
    "stream-width": "1280"
}

In depth-quality-tool i see fill rate is ~ 96%

in viewport i see "well filled" depth frame:

Now i aligned depth frame to RGB, that makes output image much
cleaner:

( Here little mess in my room i know :) )

Init stream:

rs2::context ctx;
		auto device = ctx.query_devices();
		auto dev = device[0];
		pipeline p;
		config cfg;
		string serial = dev.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER);
		cfg.enable_stream(RS2_STREAM_DEPTH, 848, 480, RS2_FORMAT_Z16, 30);
		cfg.enable_stream(RS2_STREAM_COLOR, 848, 480, RS2_FORMAT_RGB8);
		auto advanced = dev.as<advanced_mode>();
		ifstream t("C:/Users/Bumpy/source/repos/RealMotion/RealMotion/config/config.json");
		string config((istreambuf_iterator<char>(t)), istreambuf_iterator<char>());
		advanced.load_json(config);
		cfg.enable_device(serial);
		rs2::align align_to_depth(RS2_STREAM_DEPTH);
		rs2::align align_to_color(RS2_STREAM_COLOR);
		auto profile = p.start(cfg);
	

		rs2::pointcloud pc;
		rs2::points points;

		while (!QThread::currentThread()->isInterruptionRequested())
		{
		
			rs2::frameset frames = p.wait_for_frames();
			frames = align_to_color.process(frames);

			rs2::depth_frame depth = frames.get_depth_frame();
                        filter->set_filter(depth);

filter function:

void set_filter(rs2::depth_frame depth) {
		dec_filter.set_option(RS2_OPTION_FILTER_MAGNITUDE, 3);
		spat_filter.set_option(RS2_OPTION_FILTER_SMOOTH_ALPHA, 0.50f);
		spat_filter.set_option(RS2_OPTION_FILTER_MAGNITUDE, 2);
		spat_filter.set_option(RS2_OPTION_FILTER_SMOOTH_DELTA, 15);
		temp_filter.set_option(RS2_OPTION_FILTER_SMOOTH_ALPHA, 0.4f);
		temp_filter.set_option(RS2_OPTION_FILTER_SMOOTH_DELTA, 20.0f);
		rs2::disparity_transform depth_to_disparity(true);
		rs2::disparity_transform disparity_to_depth(true);
		hole_filter.set_option(RS2_OPTION_HOLES_FILL, 1);

		depth = dec_filter.process(depth);
	
		depth_to_disparity.process(depth);
		disparity_to_depth.process(depth);
		depth = spat_filter.process(depth);
		depth = hole_filter.process(depth);
	}

Validating data:

Code to show right palm 3d coordinates:

	float planarPoint3d[3];
			float pix[2] = { toSend.captured[i.key()].x, toSend.captured[i.key()].y };

			float pixel_distance_in_meters = depth.get_distance(toSend.captured[i.key()].x, toSend.captured[i.key()].y);
		
			rs2_deproject_pixel_to_point(planarPoint3d, &inrist, pix, pixel_distance_in_meters);

			QJsonObject point3D;
			point3D.insert("x", QJsonValue::fromVariant(0.01 * floor(100 * planarPoint3d[0])));
			point3D.insert("y", QJsonValue::fromVariant(0.01 * floor(100 * planarPoint3d[1])));
			point3D.insert("z", QJsonValue::fromVariant(0.01 * floor(100 * planarPoint3d[2])));
			if (i.key() == "Right palm") {
				qDebug() << "x: " << i.value().x << "y: " << i.value().y<<endl;
				qDebug() << "coords: " << point3D;
			}

Output from GIF:

coords: QJsonObject({"x":0.6900000000000001,"y":0.09,"z":1.32})
x: 734 y: 292

coords: QJsonObject({"x":0.7000000000000001,"y":0.1,"z":1.3900000000000001})
x: 723 y: 292

coords: QJsonObject({"x":0.6900000000000001,"y":0.1,"z":1.42})
x: 703 y: 292

coords: QJsonObject({"x":0.68,"y":0.1,"z":1.48})
x: 672 y: 281

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 620 y: 292

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 568 y: 281

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 517 y: 271

coords: QJsonObject({"x":0.27,"y":0.06,"z":1.74})
x: 486 y: 260

coords: QJsonObject({"x":0.18,"y":0.03,"z":1.73})
x: 455 y: 240

coords: QJsonObject({"x":0.09,"y":-0.02,"z":1.76})
x: 424 y: 240

coords: QJsonObject({"x":0,"y":-0.02,"z":1.77})
x: 392 y: 240

coords: QJsonObject({"x":-0.09,"y":-0.02,"z":1.77})
x: 372 y: 240

coords: QJsonObject({"x":-0.14,"y":-0.02,"z":1.74})
x: 351 y: 240

coords: QJsonObject({"x":-0.2,"y":-0.02,"z":1.74})
x: 341 y: 229

coords: QJsonObject({"x":-0.23,"y":-0.05,"z":1.72})
x: 320 y: 229

coords: QJsonObject({"x":-0.29,"y":-0.05,"z":1.71})
x: 299 y: 229

coords: QJsonObject({"x":-0.34,"y":-0.05,"z":1.69})
x: 279 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 258 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 229

coords: QJsonObject({"x":-0.51,"y":-0.05,"z":1.68})
x: 217 y: 229

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 196 y: 229

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 186 y: 229

coords: QJsonObject({"x":-0.6900000000000001,"y":-0.06,"z":1.78})
x: 186 y: 229

coords: QJsonObject({"x":-0.6900000000000001,"y":-0.06,"z":1.8})
x: 186 y: 229

coords: QJsonObject({"x":-0.7000000000000001,"y":-0.06,"z":1.81})
x: 186 y: 219

coords: QJsonObject({"x":-0.71,"y":-0.09,"z":1.84})
x: 196 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 206 y: 219

coords: QJsonObject({"x":-0.67,"y":-0.09,"z":1.9100000000000001})
x: 206 y: 219

coords: QJsonObject({"x":-0.68,"y":-0.09,"z":1.94})
x: 217 y: 219

coords: QJsonObject({"x":-0.65,"y":-0.09,"z":1.94})
x: 227 y: 219

coords: QJsonObject({"x":-0.63,"y":-0.09,"z":1.97})
x: 227 y: 219

coords: QJsonObject({"x":-0.64,"y":-0.1,"z":1.99})
x: 237 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 219

coords: QJsonObject({"x":-0.62,"y":-0.1,"z":2.05})
x: 237 y: 219

coords: QJsonObject({"x":-0.63,"y":-0.1,"z":2.07})
x: 237 y: 219

coords: QJsonObject({"x":-0.62,"y":-0.1,"z":2.06})
x: 237 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 229

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 240

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 250

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 227 y: 260

coords: QJsonObject({"x":-0.63,"y":0.04,"z":1.98})
x: 227 y: 260

coords: QJsonObject({"x":-0.62,"y":0.04,"z":1.94})
x: 217 y: 260

coords: QJsonObject({"x":-0.65,"y":0.04,"z":1.93})
x: 206 y: 250

coords: QJsonObject({"x":-0.68,"y":0.01,"z":1.93})
x: 196 y: 250

coords: QJsonObject({"x":-0.6900000000000001,"y":0,"z":1.86})
x: 186 y: 240

coords: QJsonObject({"x":-0.71,"y":-0.03,"z":1.85})
x: 175 y: 229

coords: QJsonObject({"x":-0.73,"y":-0.06,"z":1.8})
x: 165 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 155 y: 219

coords: QJsonObject({"x":-0.74,"y":-0.08,"z":1.7})
x: 144 y: 208

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 144 y: 208

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 144 y: 219

coords: QJsonObject({"x":-0.74,"y":-0.08,"z":1.62})
x: 155 y: 219

coords: QJsonObject({"x":-0.7000000000000001,"y":-0.08,"z":1.6})
x: 165 y: 219

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 175 y: 229

coords: QJsonObject({"x":-0.61,"y":-0.05,"z":1.51})
x: 186 y: 229

coords: QJsonObject({"x":-0.5700000000000001,"y":-0.05,"z":1.48})
x: 206 y: 229

coords: QJsonObject({"x":-0.51,"y":-0.05,"z":1.45})
x: 217 y: 229

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 237 y: 240

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 258 y: 250

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 289 y: 250

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 320 y: 271

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 351 y: 281

coords: QJsonObject({"x":0,"y":0,"z":0})
x: 392 y: 292

coords: QJsonObject({"x":-0.22,"y":0.32,"z":4.47})

PS

I think that for some reason my filter function doesnt work at all.

ev-mp · 2019-12-09T12:37:17Z

@BadMachine,
The code that you've just posted in not the same as in the original report - in the original snippet the color is aligned to depth

rs2::frameset frames = p.wait_for_frames();
frames = align_to_depth.process(frames);

while in the second post the direction is reverted with depth is aligned to color:

rs2::frameset frames = p.wait_for_frames();
frames = align_to_color.process(frames);

Thus the application flow is still not clear.
In both cases though, the critical element is still missing - the handling of RGB data.
Without seeing in code which frame is used to find tracking features it is not possible to assess whether the flow is implemented correctly.
It is obvious why aligning Depth to RGB is preferable in terms of extracting the skeleton features as the RGB image remains unchanged. Note, however that this represents a challenge as not for each RGB pixel you'll have a valid depth value, enforcing to look up for neighboring depth values.
Aligning color to depth is probably more beneficial, since even though part of the RGB data is not present in the aligned image, the pixels that do pass the filter will have 100% correspondence in the Depth data.

Please try to present the case and the issue you have in a concise and unambiguous manner, as it really not effective to try and guess the missing parts.

BadMachine · 2019-12-09T13:22:39Z

@ev-mp
Now i return to using align color to depth.
I'm really trying to make it clear.
So i want to figure out why in this frame i see "Right palm joint"

But depth of the joint pixel-point is 0?

Here is code that i wrote to catch the frame when depth of "right palm" is 0:

float planarPoint3d[3];
			float pix[2] = { toSend.captured[i.key()].x, toSend.captured[i.key()].y };

			float pixel_distance_in_meters = depth.get_distance(toSend.captured[i.key()].x, toSend.captured[i.key()].y);
		
			rs2_deproject_pixel_to_point(planarPoint3d, &inrist, pix, pixel_distance_in_meters);

			QJsonObject point3D;
			point3D.insert("x", QJsonValue::fromVariant(0.01 * floor(100 * planarPoint3d[0])));
			point3D.insert("y", QJsonValue::fromVariant(0.01 * floor(100 * planarPoint3d[1])));
			point3D.insert("z", QJsonValue::fromVariant(0.01 * floor(100 * planarPoint3d[2])));
			if (i.key() == "Right palm") {
				if (planarPoint3d[2] == 0)
				{
					imshow("image", toSend.input.getMat(ACCESS_READ));
					waitKey(30);
					//cv::imwrite("wrong depth.jpg", toSend.input);
				}

if I understand you right i should use some specific functions to get nearest pixels depth with non-zero value, right?

Also i`ll try to show you processes of my program, picture related:

Im sorry if its still unclear. Im trying to explain my problem so hard.

ev-mp · 2019-12-09T15:12:59Z

@BadMachine , thanks for the explanation.
The tracking seem quite consistent - very impressive!

For clarification :

The alignment takes place before sending the frame to pose estimator.
The pose estimator extract the tracking features from the RGB image only.
The 3d coordinates are extracted from the original depth image.
Are those assumptions correct ?

There is still something seem not right with the "Aligned Color" - I would expect that image to be much "noisier" and to have much more "zero" pixels as it is in the relevant depth image.
Can you provide the three images for verification : Original RGB, "RGB aligned to Depth" and the Depth snapshots.

When using the "Aligned RGB image make sure that the pose estimator skips the areas with no valid RGB data with zero/black pixels. And that opposing to the depth pixels where zero represents no data, you may actually get a valid RGB pixel with black color, and then you'll have to decide whether this is the color obtained from the sensor or that it received it during alignment as RGB couldn't find its correspondence in the depth frame.

BadMachine · 2019-12-09T15:56:26Z

The frames:

q: "The alignment takes place before sending the frame to pose estimator"
a: yes, ofc

q: "The pose estimator extract the tracking features from the RGB image only."
a: yes, it works only with RGB channel

q: "The 3d coordinates are extracted from the original depth image."
a: if u meant that giving depth value is from depth frame, then not sure, i guess from aligned RGB frame

float pixel_distance_in_meters = depth.get_distance(toSend.captured[i.key()].x, toSend.captured[i.key()].y);
rs2_deproject_pixel_to_point(planarPoint3d, &inrist, pix, pixel_distance_in_meters);

ev-mp · 2019-12-09T16:16:34Z

The skeleton tracking for the right hand seems to locate the palm outside of the textured (aligned) area and right into the black area. See the highlighted red dot below. To the very least - it is on the border line so it makes sense that there would be no corresponding depth data.

Once you manage to configure the estimator to move the palm's location away from the no-valid/boundary data towards the center of the mass of the right hand where it actually belongs (the convex colored shape that represents it), then IMHO you'll manage to get a consistent depth data in a reliable manner.

BadMachine · 2019-12-09T16:42:48Z

@ev-mp
Thank you for clarifying, drew attention to my problem and flattering reviews about my app architecture, really appreciate it!
Seems i should close the issue, cheers!

ev-mp · 2019-12-09T16:47:07Z

I just realized that in order to enhance depth<->color alignment required by tracking you also must ensure the temporal sync between the color and depth frames capture times is within very strict boundaries.
Using 15 FPS means that potentially there could be at least 60msec latency between color and depth (internal sync is off), so when tracking body in motion the delay affect the correspondence negatively. The ways to alleviate this is to increase FPS, set the RGB sensor's Exposure_priority control off (#5290) and verify that the actual frame rates are 30+. Enabling metadata is also recommended as it will allow to use HW rather than Host-generated timestamps (installation guide).
I would suggest to experiment with high FPS values, preferably 60 FPS at the cost of lower resolution

BadMachine · 2019-12-09T16:54:40Z

@ev-mp thanks for advice
but 15 fps - not realsense stream determinated value, it`s quite max value while using neural network for pose estimator, unfortunately)

BadMachine · 2019-12-09T20:58:07Z

@ev-mp
I think i found right way:
Using chain of post-processing filters before aligning frames, like this:

frames = frames.apply_filter(dec_filter).apply_filter(spat_filter).apply_filter(temp_filter).apply_filter(hole_filter);
auto frames_aligned = align_to_depth.process(frames);

aligned frames

Also your advice about desync frames was god damn right, i completely forgot about it. Thank u again!

ev-mp · 2019-12-10T06:36:22Z

@BadMachine , that's correct - post-processing shall be performed before alignment to avoid smearing of the generated artifacts. See a similar reply in another thread

BadMachine closed this as completed Dec 9, 2019

MartyG-RealSense mentioned this issue Jan 13, 2021

How can I back reproject RGBD data to point cloud after resizing the images? #8153

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The right way to deproject pixel to point with align to depth frame. #5403

The right way to deproject pixel to point with align to depth frame. #5403

BadMachine commented Dec 8, 2019 •

edited

Loading

BadMachine commented Dec 8, 2019

ev-mp commented Dec 9, 2019

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019 •

edited

Loading

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019 •

edited

Loading

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019

BadMachine commented Dec 9, 2019 •

edited

Loading

ev-mp commented Dec 10, 2019

The right way to deproject pixel to point with align to depth frame. #5403

The right way to deproject pixel to point with align to depth frame. #5403

Comments

BadMachine commented Dec 8, 2019 • edited Loading

Issue Description

Steps of the programm

Grabbing frames and intrinsics

Getting necessary pixels in frame

Finding 3d coordinates of those pixels

RESULT

Final questions

BadMachine commented Dec 8, 2019

ev-mp commented Dec 9, 2019

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019 • edited Loading

Init stream:

filter function:

Validating data:

Code to show right palm 3d coordinates:

Output from GIF:

PS

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019 • edited Loading

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019

ev-mp commented Dec 9, 2019

BadMachine commented Dec 9, 2019

BadMachine commented Dec 9, 2019 • edited Loading

ev-mp commented Dec 10, 2019

BadMachine commented Dec 8, 2019 •

edited

Loading

BadMachine commented Dec 9, 2019 •

edited

Loading

BadMachine commented Dec 9, 2019 •

edited

Loading

BadMachine commented Dec 9, 2019 •

edited

Loading